How do machine-generated questions compare to human-generated questions?

Lishan Zhang, Kurt VanLehn

Research output: Contribution to journalArticlepeer-review

12 Scopus citations


Science instructors need questions for use in exams, homework assignments, class discussions, reviews, and other instructional activities. Textbooks never have enough questions, so instructors must find them from other sources or generate their own questions. In order to supply biology instructors with questions for college students in introductory biology classes, two algorithms were developed. One generates questions from a formal representation of photosynthesis knowledge. The other collects biology questions from the web. The questions generated by these two methods were compared to questions from biology textbooks. Human students rated questions for their relevance, fluency, ambiguity, pedagogy, and depth. Questions were also rated by the authors according to the topic of the questions. Although the exact pattern of results depends on analytic assumptions, it appears that there is little difference in the pedagogical benefits of each class, but the questions generated from the knowledge base may be shallower than questions written by professionals. This suggests that all three types of questions may work equally well for helping students to learn.

Original languageEnglish (US)
Article number7
JournalResearch and Practice in Technology Enhanced Learning
Issue number1
StatePublished - Dec 1 2016


  • Atomic Predicate
  • Biology Student
  • Knowledge Base
  • Question Schema
  • Seed Pair

ASJC Scopus subject areas

  • Social Psychology
  • Education
  • Media Technology
  • Management of Technology and Innovation


Dive into the research topics of 'How do machine-generated questions compare to human-generated questions?'. Together they form a unique fingerprint.

Cite this