The writing pal intelligent tutoring system: Usability testing and development

Rod Roscoe, Laura K. Allen, Jennifer L. Weston, Scott A. Crossley, Danielle McNamara

Research output: Contribution to journalArticle

39 Citations (Scopus)

Abstract

The Writing Pal (W-Pal) is an intelligent tutoring system (ITS) designed to improve students' writing proficiency via a unique combination of explicit strategy instruction, game-based practice, essay writing practice, and automated formative feedback. To develop and refine the many features of the W-Pal tutoring system, we have employed a multiple-method usability testing approach, which capitalizes on the complementary strengths and weakness of methods such as focus groups, component studies, internal testing, and in vivo testing. These diverse methods allow researchers to benefit from focused student input, instructor input, and iterative development, while also gathering data in ecologically-valid settings. In this paper, we describe some of the testing and development of aspects of W-Pal, consider the challenges of building such a system, and provide a particular emphasis on a feasibility study that integrated W-Pal into high school English classrooms during a school year. The results of this study showed that students perceived the system as informative, valuable, and enjoyable, and results also highlighted specific ways that these aspects of the system could be further enhanced. Based upon these findings, a significantly updated version of W-Pal has been developed. The current W-Pal system is described along with considerations for future research and how the system may be used to supplement writing instruction.

Original languageEnglish (US)
Pages (from-to)39-59
Number of pages21
JournalComputers and Composition
Volume34
DOIs
StatePublished - Dec 1 2014

Fingerprint

Intelligent systems
Testing
Students
Feedback
writing instruction
Usability Testing
Tutoring
student
study group
school
supplement
instructor
instruction
classroom

Keywords

  • Automated writing evaluation
  • Educational games
  • Formative feedback
  • Intelligent tutoring systems
  • Usability testing
  • Writing instruction
  • Writing strategies

ASJC Scopus subject areas

  • Language and Linguistics
  • Education
  • Linguistics and Language
  • Computer Science(all)

Cite this

The writing pal intelligent tutoring system : Usability testing and development. / Roscoe, Rod; Allen, Laura K.; Weston, Jennifer L.; Crossley, Scott A.; McNamara, Danielle.

In: Computers and Composition, Vol. 34, 01.12.2014, p. 39-59.

Research output: Contribution to journalArticle

Roscoe, Rod ; Allen, Laura K. ; Weston, Jennifer L. ; Crossley, Scott A. ; McNamara, Danielle. / The writing pal intelligent tutoring system : Usability testing and development. In: Computers and Composition. 2014 ; Vol. 34. pp. 39-59.
@article{caa738aac2084a9c9bb8a96a16b4eddb,
title = "The writing pal intelligent tutoring system: Usability testing and development",
abstract = "The Writing Pal (W-Pal) is an intelligent tutoring system (ITS) designed to improve students' writing proficiency via a unique combination of explicit strategy instruction, game-based practice, essay writing practice, and automated formative feedback. To develop and refine the many features of the W-Pal tutoring system, we have employed a multiple-method usability testing approach, which capitalizes on the complementary strengths and weakness of methods such as focus groups, component studies, internal testing, and in vivo testing. These diverse methods allow researchers to benefit from focused student input, instructor input, and iterative development, while also gathering data in ecologically-valid settings. In this paper, we describe some of the testing and development of aspects of W-Pal, consider the challenges of building such a system, and provide a particular emphasis on a feasibility study that integrated W-Pal into high school English classrooms during a school year. The results of this study showed that students perceived the system as informative, valuable, and enjoyable, and results also highlighted specific ways that these aspects of the system could be further enhanced. Based upon these findings, a significantly updated version of W-Pal has been developed. The current W-Pal system is described along with considerations for future research and how the system may be used to supplement writing instruction.",
keywords = "Automated writing evaluation, Educational games, Formative feedback, Intelligent tutoring systems, Usability testing, Writing instruction, Writing strategies",
author = "Rod Roscoe and Allen, {Laura K.} and Weston, {Jennifer L.} and Crossley, {Scott A.} and Danielle McNamara",
year = "2014",
month = "12",
day = "1",
doi = "10.1016/j.compcom.2014.09.002",
language = "English (US)",
volume = "34",
pages = "39--59",
journal = "Computers and Composition",
issn = "8755-4615",
publisher = "Elsevier Limited",

}

TY - JOUR

T1 - The writing pal intelligent tutoring system

T2 - Usability testing and development

AU - Roscoe, Rod

AU - Allen, Laura K.

AU - Weston, Jennifer L.

AU - Crossley, Scott A.

AU - McNamara, Danielle

PY - 2014/12/1

Y1 - 2014/12/1

N2 - The Writing Pal (W-Pal) is an intelligent tutoring system (ITS) designed to improve students' writing proficiency via a unique combination of explicit strategy instruction, game-based practice, essay writing practice, and automated formative feedback. To develop and refine the many features of the W-Pal tutoring system, we have employed a multiple-method usability testing approach, which capitalizes on the complementary strengths and weakness of methods such as focus groups, component studies, internal testing, and in vivo testing. These diverse methods allow researchers to benefit from focused student input, instructor input, and iterative development, while also gathering data in ecologically-valid settings. In this paper, we describe some of the testing and development of aspects of W-Pal, consider the challenges of building such a system, and provide a particular emphasis on a feasibility study that integrated W-Pal into high school English classrooms during a school year. The results of this study showed that students perceived the system as informative, valuable, and enjoyable, and results also highlighted specific ways that these aspects of the system could be further enhanced. Based upon these findings, a significantly updated version of W-Pal has been developed. The current W-Pal system is described along with considerations for future research and how the system may be used to supplement writing instruction.

AB - The Writing Pal (W-Pal) is an intelligent tutoring system (ITS) designed to improve students' writing proficiency via a unique combination of explicit strategy instruction, game-based practice, essay writing practice, and automated formative feedback. To develop and refine the many features of the W-Pal tutoring system, we have employed a multiple-method usability testing approach, which capitalizes on the complementary strengths and weakness of methods such as focus groups, component studies, internal testing, and in vivo testing. These diverse methods allow researchers to benefit from focused student input, instructor input, and iterative development, while also gathering data in ecologically-valid settings. In this paper, we describe some of the testing and development of aspects of W-Pal, consider the challenges of building such a system, and provide a particular emphasis on a feasibility study that integrated W-Pal into high school English classrooms during a school year. The results of this study showed that students perceived the system as informative, valuable, and enjoyable, and results also highlighted specific ways that these aspects of the system could be further enhanced. Based upon these findings, a significantly updated version of W-Pal has been developed. The current W-Pal system is described along with considerations for future research and how the system may be used to supplement writing instruction.

KW - Automated writing evaluation

KW - Educational games

KW - Formative feedback

KW - Intelligent tutoring systems

KW - Usability testing

KW - Writing instruction

KW - Writing strategies

UR - http://www.scopus.com/inward/record.url?scp=84907954890&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84907954890&partnerID=8YFLogxK

U2 - 10.1016/j.compcom.2014.09.002

DO - 10.1016/j.compcom.2014.09.002

M3 - Article

AN - SCOPUS:84907954890

VL - 34

SP - 39

EP - 59

JO - Computers and Composition

JF - Computers and Composition

SN - 8755-4615

ER -