Confidence Weighted Multiple-Choice Questioning

In the United States, the multiple-choice question is probably the most prolific type of question students encounter. As early as kindergarten, learners are made to choose among responses A, B, or C. This sort of responding to a number of choices doesn’t end for most until they are through with university schooling. But, just because something is popular doesn’t mean it is good. Multiple-choice questioning is not without its flaws. One of the main criticisms is that it allows for guessing. If a student doesn’t know whether to answer A, B, C, or D, they’ve still got a 25% chance of answering correctly by simply just choosing one of the four possible answer responses. That sort of luck isn’t really indicative of a reliable or valid assessment.

For some, that reason alone is enough to make them write off the multiple-choice format all together. I believe that is a mistake. Learners will continually be placed in situations where they will encounter these questions, so rather than do away with them, I’ve kinda taken it upon myself to improve the multiple-choice format. While its use as an assessment of learning should reasonably be called into question, its ability to be useful as an assessment for/as learning is less dubious. 

I have previously written about how I use the multiple-choice format in, what I believe, is a more thoughtful and useful manner for learning:

1. Ranking Multiple-Choice Responses to Increase Cognition
2. Maximizing the Effectiveness of Multiple-Choice Questions

Without going into too much detail, these strategies require students to effortfully consider and work with all responses provided for a particular multiple-choice question. Providing these sorts of tasks for learners not only leads to more desirable difficulties, but also assists with creating more healthy study habits. Generally speaking, more pensive consideration and application of material provides better opportunities for retention of information long-term. Getting students in the habit of taking a closer look at all responses provided for a multiple-choice question not only may lead to fewer careless mistakes, but can also genuinely change how students cognitively approach these questions.

I’d like to discuss and explain another strategy that improves upon the standard multiple-choice question: the confidence weighted multiple-choice format (1). Like the two methods mentioned in the previous paragraph, this strategy requires students to do more than simply choose the correct answer. Learners are tasked with careful consideration of the confidence in the correct answer. If they are not completely confident, there are options to indicate this uncertainty.

Here is an example of what this looked like during experimentation:

If a student is completely confident the answer is Venus (it is), they should indicate this by choosing the circle directly next to Venus. If, however, a student is less confident and is unsure whether it is Venus or Mercury, they would choose one of the circles between those answer choices. If they choose the circle closer to Venus than Mercury, this indicates they are leaning more closely to Venus being the correct answer, but cannot completely rule out Mercury. Choosing the circle in the middle of Venus and Mercury, indicates they equally believe Venus and Mercury could be the correct answer. There is also a ‘don’t know’ option if they have no idea. 

The researchers also added point values to the students’ choices, rewarding the correct answer with the most points and providing for negative points if the incorrect answer is chosen. For those students who are quite competitive, this might also add to their levels of focus and consideration. 

I will admit that I’m not a big fan of adding points, superficial or not, to retrieval activities like this. It may add to student anxiety and, honestly, accruing points wouldn’t be the goal of using the confidence weighted multiple-choice format in my classroom. I want, instead, for learners to focus on their thinking and consideration of material while using this particular format. I really want my students to appreciate learning for the sake of learning, and not so much for an assigned grade. 

The current study shows evidence of improved performance on a final cued-recall test when using the confidence weighted multiple-choice format versus a standard multiple-choice format group and a study only group when tested on related information. 

This seems to indicate that using the confidence weighted multiple-choice format not only assists students with understanding the correct answer, but also related information associated with the incorrect answer choices. This is where the potential lies for differing multiple-choice strategies that improve upon the standard multiple-choice format. Not only do these enhanced formats indicate the correct answer for students, they also assist students with retrieving information associated with all response choices. By providing opportunities for learners to thoughtfully consider all material for a particular multiple-choice question, students are now interacting with more of the subject matter; comparing and contrasting, associating, differentiating, et cetera. And again, generally speaking, more retrieval and cognition about and with the material will lead to more retention of material…really powerful stuff.

Now, of course, nothing is perfect. As a classroom teacher myself, the biggest potential negative of using the confidence weighted multiple-choice questioning is time consumption using the format. This is certainly not something I would have my students use with 20+ questions. I’m thinking more like 3-5 in this format to start class or review most missed questions on a prior assessment. It will also take some time commitment to explain this strategy and practice with students. But, that is the case with any new activity done in class. Over the course of a semester, students will become more familiar with this and the time used to complete a task using the confidence weighted format will decrease. While there may be a significant amount of time required to make the triangles for students to use and explain and implement the strategy in class, I see this as a worthwhile investment. Anytime I can provide my students with a method of retrieval and assessment that improves retention of material while also modeling proper study habits, that’s a big win. 

Along with the two strategies mentioned earlier, I have made a poster describing their implementation and research on construction of multiple-choice questions. Please feel free to download and use however you see fit.

---

Feature Image by Elijah Hail on Unsplash