East Carolina University
Department of Psychology
There are a number of tactics you can use to do better on a poorly constructed multiple choice test than you would do without employing such tactics. Employing these tactics should result in your doing better than chance-level even if you know nothing about the material being tested.
It usually takes more words to state a correct answer than to state an incorrect answer. Knowing this, I try to make long some of the wrong answer options on my multiple choice items. I want to measure how much my students have learned about the content of my courses, not what test taking tactics they have learned. Variance in test scores due to individual differences in test taking skills is, IMHO, error variance, and I do what I can to reduce such error.
Several years ago Kalat (at N. C. State) demonstrated that this technique works well with test items provided by the publishers of introductory psychology text books. In March of 2002 I tried this "longest answer" technique with the exam I gave in my undergraduate statistics class. Each question had four answer options, so doing better than chance would be better than 25% correct. The longest answer tactic failed miserably. On only 5% of the items was the longest answer correct. It seems that I may actually overcorrect for the tendency for longer answers to be correct.
The instructor is observed looking over the test after she has distributed it, and then she announces something like "Class, look at item number 27. There is a typo in answer option c -- it should read ......" In this case, you should seriously consider choosing the corrected option. When reading over an exam, the instructor's eyes will be drawn to the correct answer options, which she will likely read carefully, finding any typos there, but she probably will not read the incorrect answer options so carefully.
A student comes up and speaks to the instructor, after which the instructor announces that there is a typo in one of the answer options. Since the instructor did not find the typo, this answer option is probably wrong.
It is not unusual for one to be able to find elsewhere on the exam the information necessary to answer the question at hand.
Most instructors won't make the same option correct on more than two or three successive questions (for example, c, c, c, c). On the other hand, if the test maker uses a random process to determine the position of the correct answer option on each item (as do I), it is not at all unusual to get "runs" of three or more.
Naive test writers don't even consider an answer option like "all of the above," "two of the above," or "none of the above" unless that is the correct answer. Accordingly, one should be expected to do better than chance simply by choosing "all of the above" or a similar answer option whenever it occurs. I have attempted to eliminate this source of error in my multiple choice items by remembering to include answers like "all of the above" when that answer option is not correct. I warn my students of this, suggesting that they not simply elect "all of the above" every time that is one of the options.
One of the students who took the March, 2002, statistics exam thought I was misleading students by telling them that I include answer options like "all of the above" even when they are not the correct answer. Apparently she interpreted my warning to mean "never choose all of the above." I should have made it clear that writing some questions where "all of the above" is an incorrect answer option does not exclude also writing other questions where "all of the above" is the correct answer option. One of my colleagues (Jon Reed) told me that his students have decided that he uses "all of the above" so often when it is not the correct answer that they use a strategy of never choosing "all of the above." That strategy may, or may not, be effective on Dr. Reed's examinations.
As a test of the "choose all of the above" tactic on the statistics exam I gave in March of 2002, I segregated out the items that included an answer option such as "all of the above." The tactic successfully identified the correct answer in 5 of the 11 questions (45%). While this does suggest that one would do better than chance by just always choosing "all of the above," nobody is going to pass an exam getting only 45% of the questions correct. Students applying this strategy on this exam would do more poorly than the average student.
Perhaps I should try to include even more "all of the above" answers when that answer is incorrect. Of course, sampling error may have contributed to the 45% success rate for the tactic on the sample of items I used to evaluate the tactic. If I were successful in eliminating all error due to the use of this tactic, then the probability of success would be 25% each time the tactic was employed. A 95% confidence interval for the success rate extends from 19% to 71%, which includes the chance level 25%, so the observed success rate does not differ significantly from that which would be expected by chance.
Of course, it helps to know the material that is being tested. My standard recommendation is to distribute your practice and not to cram the night before the exam. In class you should listen to understand, not try to transcribe everything the teacher says. As soon as possible after class, reproduce the contents of the class in your own words. Note anything that is not clear, and prepare to ask the teacher about it during the next class. Pretend that you are going to be guest lecturing on the topic covered in today's class and prepare your lecture note. Also prepare some examination questions covering that material.
If you have tips that I have not mentioned here, please let me know about them and I'll consider adding them.
Contact Information for the Webmaster,
Dr. Karl L. Wuensch
This page most recently revised on
13. July 2009.