Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies

Min Chi, Kurt VanLehn, Diane Litman, Pamela Jordan

Research output: Contribution to journalArticle

52 Citations (Scopus)

Abstract

For many forms of e-learning environments, the system's behavior can be viewed as a sequential decision process wherein, at each discrete step, the system is responsible for selecting the next action to take. Pedagogical strategies are policies to decide the next system action when there are multiple ones available. In this project we present a Reinforcement Learning (RL) approach for inducing effective pedagogical strategies and empirical evaluations of the induced strategies. This paper addresses the technical challenges in applying RL to Cordillera, a Natural Language Tutoring System teaching students introductory college physics. The algorithm chosen for this project is a model-based RL approach, Policy Iteration, and the training corpus for the RL approach is an exploratory corpus, which was collected by letting the system make random decisions when interacting with real students. Overall, our results show that by using a rather small training corpus, the RL-induced strategies indeed measurably improved the effectiveness of Cordillera in that the RL-induced policies improved students' learning gains significantly.

Original languageEnglish (US)
Pages (from-to)137-180
Number of pages44
JournalUser Modeling and User-Adapted Interaction
Volume21
Issue number1-2
DOIs
StatePublished - Apr 2011

Fingerprint

Reinforcement learning
induction
reinforcement
learning
Students
policy approach
student
learning strategy
electronic learning
physics
Teaching
learning environment
Physics
language
evaluation

Keywords

  • Human learning
  • Machine learning
  • Pedagogical strategy
  • Reinforcement learning

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Science Applications
  • Education

Cite this

Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. / Chi, Min; VanLehn, Kurt; Litman, Diane; Jordan, Pamela.

In: User Modeling and User-Adapted Interaction, Vol. 21, No. 1-2, 04.2011, p. 137-180.

Research output: Contribution to journalArticle

@article{89c8c141f3f94b318af8d8db0bc3ab9c,
title = "Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies",
abstract = "For many forms of e-learning environments, the system's behavior can be viewed as a sequential decision process wherein, at each discrete step, the system is responsible for selecting the next action to take. Pedagogical strategies are policies to decide the next system action when there are multiple ones available. In this project we present a Reinforcement Learning (RL) approach for inducing effective pedagogical strategies and empirical evaluations of the induced strategies. This paper addresses the technical challenges in applying RL to Cordillera, a Natural Language Tutoring System teaching students introductory college physics. The algorithm chosen for this project is a model-based RL approach, Policy Iteration, and the training corpus for the RL approach is an exploratory corpus, which was collected by letting the system make random decisions when interacting with real students. Overall, our results show that by using a rather small training corpus, the RL-induced strategies indeed measurably improved the effectiveness of Cordillera in that the RL-induced policies improved students' learning gains significantly.",
keywords = "Human learning, Machine learning, Pedagogical strategy, Reinforcement learning",
author = "Min Chi and Kurt VanLehn and Diane Litman and Pamela Jordan",
year = "2011",
month = "4",
doi = "10.1007/s11257-010-9093-1",
language = "English (US)",
volume = "21",
pages = "137--180",
journal = "User Modeling and User-Adapted Interaction",
issn = "0924-1868",
publisher = "Springer Netherlands",
number = "1-2",

}

TY - JOUR

T1 - Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies

AU - Chi, Min

AU - VanLehn, Kurt

AU - Litman, Diane

AU - Jordan, Pamela

PY - 2011/4

Y1 - 2011/4

N2 - For many forms of e-learning environments, the system's behavior can be viewed as a sequential decision process wherein, at each discrete step, the system is responsible for selecting the next action to take. Pedagogical strategies are policies to decide the next system action when there are multiple ones available. In this project we present a Reinforcement Learning (RL) approach for inducing effective pedagogical strategies and empirical evaluations of the induced strategies. This paper addresses the technical challenges in applying RL to Cordillera, a Natural Language Tutoring System teaching students introductory college physics. The algorithm chosen for this project is a model-based RL approach, Policy Iteration, and the training corpus for the RL approach is an exploratory corpus, which was collected by letting the system make random decisions when interacting with real students. Overall, our results show that by using a rather small training corpus, the RL-induced strategies indeed measurably improved the effectiveness of Cordillera in that the RL-induced policies improved students' learning gains significantly.

AB - For many forms of e-learning environments, the system's behavior can be viewed as a sequential decision process wherein, at each discrete step, the system is responsible for selecting the next action to take. Pedagogical strategies are policies to decide the next system action when there are multiple ones available. In this project we present a Reinforcement Learning (RL) approach for inducing effective pedagogical strategies and empirical evaluations of the induced strategies. This paper addresses the technical challenges in applying RL to Cordillera, a Natural Language Tutoring System teaching students introductory college physics. The algorithm chosen for this project is a model-based RL approach, Policy Iteration, and the training corpus for the RL approach is an exploratory corpus, which was collected by letting the system make random decisions when interacting with real students. Overall, our results show that by using a rather small training corpus, the RL-induced strategies indeed measurably improved the effectiveness of Cordillera in that the RL-induced policies improved students' learning gains significantly.

KW - Human learning

KW - Machine learning

KW - Pedagogical strategy

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=79955803320&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79955803320&partnerID=8YFLogxK

U2 - 10.1007/s11257-010-9093-1

DO - 10.1007/s11257-010-9093-1

M3 - Article

AN - SCOPUS:79955803320

VL - 21

SP - 137

EP - 180

JO - User Modeling and User-Adapted Interaction

JF - User Modeling and User-Adapted Interaction

SN - 0924-1868

IS - 1-2

ER -