Ranking Multiple-Choice Answers to Increase Cognition

Multiple-choice questions…some people love them, some people hate them. I, for the most part, fall into the first category. I believe they are helpful for learning and remembering for a multitude of reasons that I won’t bore you with at the moment. However, I do understand some of the judgment with this style of questioning: students may guess the correct answer, students are only attempting to recognize the correct answer instead of recalling the answer, et cetera. I get it. This criticism primarily led with my first offering to improve multiple-choice questions, Maximizing the Effectiveness of Multiple Choice Questions, which is one of my more popular blog posts. The strategy employed with the above article requires students to not only choose a correct answer, but use all of the answer choices in some form. This eliminates a bit of guessing and tasks students with thoughtfully considering and recalling information learned to successfully complete the assignment. 

While having a look at Atkinson & Shiffrin (1968) this week, I came across an interesting study (Binford and Gettys (1965)) that asked subjects to choose a correct answer from given options. If the subject answered correctly, they moved on to the next questions. If they answered incorrectly, they were to attempt to answer again from the remaining options. They found that subjects’ second attempt resulted in a correct answer well above the chance level to be expected. Now, this isn’t the focus of the research, but rather an interesting aspect of their work…and it got me thinking. What if the subjects read through the options and, perhaps unconsciously, ranked them from most correct to least correct? Following that line of thinking, what if the subjects next chose the option they ranked as most correct to verbalize as their answer? When they were found to be incorrect, did they simply move to the option they ranked as second most correct?

All of these questions swirled around in my head and then it hit me…what if students were made to make this somewhat covert thinking quite overt? 

What if, when presented with a multiple-choice question, students had to not only choose the correct answer, but they had to rank each option from most correct to somewhat correct to somewhat incorrect to most incorrect? Then, after ranking each answer choice, the students are tasked with explaining why the option is correct or incorrect AND why they ranked one answer choice above or below another. In my mind, students are comparing and contrasting elements of each option and considering the most important aspects of the question stem. I see this as a strategy to get students not only retrieving material to answer questions, but also integrating different details of the answer choices, thereby assimilating and accommodating existing schemas

This can easily turn into a lively discussion between partners or among an entire class. I can envision a nice debate weighing out the rankings of each question…why Jasmine placed answer choice ‘B’ ahead of ‘D’ and why Fatima disagrees. Students are not only given the opportunity to inject their opinion, but are tasked with confronting evidence to the contrary. I think it’s a winner. And, given the move to online learning due to the COVID-19 pandemic, this entire strategy can easily adapt to the online classroom environment. 

I’m not exactly sure how I want my students to encounter this strategy. Here’s one option that is very explicit, with more instruction and organization than option two, but it isn’t as easily adaptable. I can also see space and formatting becoming a problem, depending on the length of the question and the student’s answers.

Option two is simpler and more malleable to different questions, but doesn’t provide as much organization or structure.

I’m not sure which will work better or if I’ll scrap both and come up with something better. As I’m just formulating how I’ll do this in class as I write, I’m not too sure what will work. Your suggestions are always welcome. 🙂

In the end, I appreciate the multiple-choice question. When done correctly, they can provide ample opportunities for students to think with and about subject material at a high level. I hope this can be a strategy you use in your classroom to allow your students to really study, ponder, and evaluate material. Obviously, feel free to accommodate and adjust any aspects to fit your students. 

What do you see that can be improved upon with this strategy?

How might you change some aspects to fit your classroom?

Feature image by Green Chameleon on Unsplash

3 thoughts on “Ranking Multiple-Choice Answers to Increase Cognition

Add yours

  1. I love this and will adapt a version of it. I used to use modified multiple choice where students would have to explain why the answer they chose was the correct one (process of elimination is not an explanation). This is a more structured version.

    Although I KNOW I would have to change the categories. I can already hear my student arguing that “somewhat correct” is the same thing as “somewhat incorrect” or asking what’s the difference? I would use something like “More correct than wrong” and “More Wrong than correct” to help them make the distinctions.

    That said, I find the option 2 cleaner and easier to follow (and grade) but I would add the column where they enter the choice: something like

    Ans—-ranking——————Explanation
    B——–Correct—————–This fits with the theorist definition of….
    A More correct
    than wrong———–This follows the defintion but leaves out
    D More wrong
    than correct———-This clearly violates the defintion but does mention….
    C Wrong——————This meets none of the criteria for the Theorist’s position that….

    I really can’t imagine more than 5 questions like this on an exam. It could easily be open book and open not as well.

    Thanks for the suggestion.

Leave a Reply

Up ↑

%d bloggers like this: