A Commentary on Construct Validity When Using Operational Virtual Learning Environment Data in Effectiveness Studies

A. Corinne Huggins-Manley, Carole R. Beal, Sidney K. D’Mello, Walter L. Leite, Dyugu Dee Cetin-Berber, Dongho Kim, Danielle S. McNamara

Research output: Contribution to journalArticle

Abstract

Virtual learning environments (VLEs) are increasingly used at-scale in educational contexts to facilitate teaching and promote learning, and the data they produce can be used for educational research purposes. Meanwhile, the U.S. Department of Education’s Office of Educational Technology has repeatedly emphasized the importance of using evidence to validate claims from VLE-based educational research. Although VLE data can provide some affordances for conducting educational research, we argue that many challenges can arise with respect to providing evidence for construct validity. The objective of this commentary is to encourage educational researchers using operational, at-scale VLE data to align their data and intended constructs to a theoretical framework of construct validity threats in order to develop a comprehensive set of actionable solutions. We use examples from our research project as a demonstration resource for performing such an alignment.

Original languageEnglish (US)
Pages (from-to)750-759
Number of pages10
JournalJournal of Research on Educational Effectiveness
Volume12
Issue number4
DOIs
StatePublished - Oct 2 2019

Fingerprint

construct validity
learning environment
educational research
educational technology
evidence
research project
threat
Teaching
resources
learning
education

Keywords

  • construct validity
  • educational technology
  • measurement
  • virtual learning environments

ASJC Scopus subject areas

  • Education

Cite this

A Commentary on Construct Validity When Using Operational Virtual Learning Environment Data in Effectiveness Studies. / Huggins-Manley, A. Corinne; Beal, Carole R.; D’Mello, Sidney K.; Leite, Walter L.; Cetin-Berber, Dyugu Dee; Kim, Dongho; McNamara, Danielle S.

In: Journal of Research on Educational Effectiveness, Vol. 12, No. 4, 02.10.2019, p. 750-759.

Research output: Contribution to journalArticle

Huggins-Manley, A. Corinne ; Beal, Carole R. ; D’Mello, Sidney K. ; Leite, Walter L. ; Cetin-Berber, Dyugu Dee ; Kim, Dongho ; McNamara, Danielle S. / A Commentary on Construct Validity When Using Operational Virtual Learning Environment Data in Effectiveness Studies. In: Journal of Research on Educational Effectiveness. 2019 ; Vol. 12, No. 4. pp. 750-759.
@article{b94b4142355a42d99251ba13ee3fbb03,
title = "A Commentary on Construct Validity When Using Operational Virtual Learning Environment Data in Effectiveness Studies",
abstract = "Virtual learning environments (VLEs) are increasingly used at-scale in educational contexts to facilitate teaching and promote learning, and the data they produce can be used for educational research purposes. Meanwhile, the U.S. Department of Education’s Office of Educational Technology has repeatedly emphasized the importance of using evidence to validate claims from VLE-based educational research. Although VLE data can provide some affordances for conducting educational research, we argue that many challenges can arise with respect to providing evidence for construct validity. The objective of this commentary is to encourage educational researchers using operational, at-scale VLE data to align their data and intended constructs to a theoretical framework of construct validity threats in order to develop a comprehensive set of actionable solutions. We use examples from our research project as a demonstration resource for performing such an alignment.",
keywords = "construct validity, educational technology, measurement, virtual learning environments",
author = "Huggins-Manley, {A. Corinne} and Beal, {Carole R.} and D’Mello, {Sidney K.} and Leite, {Walter L.} and Cetin-Berber, {Dyugu Dee} and Dongho Kim and McNamara, {Danielle S.}",
year = "2019",
month = "10",
day = "2",
doi = "10.1080/19345747.2019.1639869",
language = "English (US)",
volume = "12",
pages = "750--759",
journal = "Journal of Research on Educational Effectiveness",
issn = "1934-5747",
publisher = "Routledge",
number = "4",

}

TY - JOUR

T1 - A Commentary on Construct Validity When Using Operational Virtual Learning Environment Data in Effectiveness Studies

AU - Huggins-Manley, A. Corinne

AU - Beal, Carole R.

AU - D’Mello, Sidney K.

AU - Leite, Walter L.

AU - Cetin-Berber, Dyugu Dee

AU - Kim, Dongho

AU - McNamara, Danielle S.

PY - 2019/10/2

Y1 - 2019/10/2

N2 - Virtual learning environments (VLEs) are increasingly used at-scale in educational contexts to facilitate teaching and promote learning, and the data they produce can be used for educational research purposes. Meanwhile, the U.S. Department of Education’s Office of Educational Technology has repeatedly emphasized the importance of using evidence to validate claims from VLE-based educational research. Although VLE data can provide some affordances for conducting educational research, we argue that many challenges can arise with respect to providing evidence for construct validity. The objective of this commentary is to encourage educational researchers using operational, at-scale VLE data to align their data and intended constructs to a theoretical framework of construct validity threats in order to develop a comprehensive set of actionable solutions. We use examples from our research project as a demonstration resource for performing such an alignment.

AB - Virtual learning environments (VLEs) are increasingly used at-scale in educational contexts to facilitate teaching and promote learning, and the data they produce can be used for educational research purposes. Meanwhile, the U.S. Department of Education’s Office of Educational Technology has repeatedly emphasized the importance of using evidence to validate claims from VLE-based educational research. Although VLE data can provide some affordances for conducting educational research, we argue that many challenges can arise with respect to providing evidence for construct validity. The objective of this commentary is to encourage educational researchers using operational, at-scale VLE data to align their data and intended constructs to a theoretical framework of construct validity threats in order to develop a comprehensive set of actionable solutions. We use examples from our research project as a demonstration resource for performing such an alignment.

KW - construct validity

KW - educational technology

KW - measurement

KW - virtual learning environments

UR - http://www.scopus.com/inward/record.url?scp=85076397934&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85076397934&partnerID=8YFLogxK

U2 - 10.1080/19345747.2019.1639869

DO - 10.1080/19345747.2019.1639869

M3 - Article

AN - SCOPUS:85076397934

VL - 12

SP - 750

EP - 759

JO - Journal of Research on Educational Effectiveness

JF - Journal of Research on Educational Effectiveness

SN - 1934-5747

IS - 4

ER -