Abstract
Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called “distractors”). We designed an algorithm to generate distractors from a semantic network for four types of multiple choice questions in biology. By recruiting 200 participants from Amazon Mechanical Turk, the machine-generated distractors were compared to human-generated distractors in terms of question difficulty, question discrimination and distractor usefulness. The machine-generated and human-generated distractors performed very closely on all the three measures, suggesting that generating distractors from a semantic network for simple multiple choice questions is a viable method.
| Original language | English (US) |
|---|---|
| Journal | Interactive Learning Environments |
| DOIs | |
| State | Published - Jan 1 2019 |
Fingerprint
Keywords
- crowd sourcing
- distractor evaluation
- item analysis
- multiple choice question
- Question generation
- semantic network
ASJC Scopus subject areas
- Education
- Computer Science Applications
Cite this
Evaluation of auto-generated distractors in multiple choice questions from a semantic network. / Zhang, Lishan; VanLehn, Kurt.
In: Interactive Learning Environments, 01.01.2019.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Evaluation of auto-generated distractors in multiple choice questions from a semantic network
AU - Zhang, Lishan
AU - VanLehn, Kurt
PY - 2019/1/1
Y1 - 2019/1/1
N2 - Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called “distractors”). We designed an algorithm to generate distractors from a semantic network for four types of multiple choice questions in biology. By recruiting 200 participants from Amazon Mechanical Turk, the machine-generated distractors were compared to human-generated distractors in terms of question difficulty, question discrimination and distractor usefulness. The machine-generated and human-generated distractors performed very closely on all the three measures, suggesting that generating distractors from a semantic network for simple multiple choice questions is a viable method.
AB - Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called “distractors”). We designed an algorithm to generate distractors from a semantic network for four types of multiple choice questions in biology. By recruiting 200 participants from Amazon Mechanical Turk, the machine-generated distractors were compared to human-generated distractors in terms of question difficulty, question discrimination and distractor usefulness. The machine-generated and human-generated distractors performed very closely on all the three measures, suggesting that generating distractors from a semantic network for simple multiple choice questions is a viable method.
KW - crowd sourcing
KW - distractor evaluation
KW - item analysis
KW - multiple choice question
KW - Question generation
KW - semantic network
UR - http://www.scopus.com/inward/record.url?scp=85066906007&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85066906007&partnerID=8YFLogxK
U2 - 10.1080/10494820.2019.1619586
DO - 10.1080/10494820.2019.1619586
M3 - Article
AN - SCOPUS:85066906007
JO - Interactive Learning Environments
JF - Interactive Learning Environments
SN - 1049-4820
ER -