Multiple-choice Items That Measure Up

They find themselves suddenly in the weeds of communicating clearly and assessment validity (making sure items measure what is intended to be measured).  Clear multiple-choice items measuring accurately students’ knowledge and skill applications take time and elbow grease to develop.  Here are some suggestions to help you write quality multiple-choice items and to save some of your valuable preparation time.

Am I Measuring What I Want To Measure?

Quick-reference lists can help you improve item and overall assessment validity.  Refer to your unit plan or lessons for essential questions and enduring understandings.  List these using a few key words for each.  Do the same for skills (behaviors).   These lists represent your measurable content and skills.  Check-off the list items as you address each with one or more multiple-choice questions.   This will ensure each question addresses specific content and skills and that you cover all of what you need to cover.  A few moments devoted to creating lists of what is to be measured and connecting drafted items to these lists can improve assessment validity and your efficiency.

Stems and Options

Each multiple-choice item consists of a stem and a set of response options: one correct response and multiple incorrect responses.  An item’s structure can be just as important as the clarity of your writing.

Responses need to be shorter than the stem so the item’s emphasis is on the content or skill and not the student’s reading abilities or ability to hold information in short term memory.  Consider the following example…

Students reading the options may forget or confuse the purpose established in the stem.  Could the student inadvertently switch “the ask” from push to pull while reading through the options?  Is the item intended to measure a student’s memory, reading ability, or ability to differentiate between push and pull factors?  Consider the following alternative…

This rewrite asked the same question with less than half the number of words, and most of the words established the item’s purpose in the stem.  Social Studies rockstars knowing the differences between the factors of migration will be able to demonstrate what they know even if they are struggling readers or are still developing short-term memory.

Be Mindful of Your Options

Creating clear correct options and purposeful detractors (incorrect options) are equally important.  Let’s presume you are writing a traditional four-option item:

1) Keep your correct option free of unnecessary context clues linking it to the stem.  Avoid using shared language— key terms— which can clue students into choosing the correct option without actually knowing the information or skill being measured.

2) Keep your options succinct and free of possible misinterpretations.  Fewer words can mean fewer ways to misinterpret the option.

3) Two of your three incorrect options should be reasonable and about the same length as the correct option. A popular student “trick” for eliminating options when unsure of the tested material is to avoid the longer ones, which have a tendency to be incorrect.

4) Creating a “left field” response for the fourth option is helpful when trying to determine the effectiveness of instruction and students’ preparations.  A large number of students choosing this option for a given item is usually a red flag regarding the teaching of the measured content or skill.  A student choosing a significant number of left field options within a given assessment indicates the student is experiencing some significant confusion, dealing with an issue unrelated to the class interfering with performance, or needs some coaching in regard to study habits. Here is an example showing purposeful options…

Well-written and purposeful options increase assessment validity by avoiding misinterpretations and confusion, deemphasize the impact of test-taking tricks, and create information-rich windows into the effectiveness of instruction and students’ understandings.

Take The Assessment

Place yourself in your student’s chair and take the assessment.  This gives you a slightly different perspective when editing and will help you find errors, everything from misplaced commas to missing answer options— sometimes even the correct options.

Look for overused letters and patterns.  The numbers of “a’s,” “b’s,” “c’s,” and “d’s” should be roughly the same.  A graduate school professor shared with me decades ago that test writers tend to select the letter “b” more often than other options when drafting tests.  I have found this to be true time and again when checking my assessment drafts.  Strange, but true.  Patterns can also influence students to choose particular options, particularly if a student is unsure of the answer, and can even enable cheating.

“Hey Billy, was Mr. Cooper’s test hard?  I need to take it after lunch.”

“It was easy.  Just choose ‘a,’ ‘c’, and ’d’ in a row three times for the first nine questions.”

I mean, we at least need to make cheating as challenging as studying, right?

A Final Check Using a Panel

Researchers developing measurement tools (like school assessments) often conduct panel reviews to proofread and ensure validity.  You can apply this practice quickly and informally to check your multiple-choice (or, any type of) items.  Add a key to your original content and skills lists assigning a letter, number, or color-code to each list-item.  Provide a few colleagues with your lists and copies of your assessment.  Ask your colleagues to take the assessment, identify any errors, and offer revision ideas in the margins.  Finally, ask them to label each item according to your key.  Did your colleagues identify the content and/or skill you had intended for each multiple-choice item?  Can you use any of the edit and revision suggestions to make your items more clear, concise, and/or accurate?  An extra set of trained eyes can help to ensure validity, identify errors, and offer revision suggestions.