Use of internal consistency coefficients for estimating reliability of experimental task scores

Samuel B. Green, Yanyun Yang, Mary Alt, Shara Brinkley, Shelley Gray, Tiffany Hogan, Nelson Cowan

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks.

Original languageEnglish (US)
JournalPsychonomic Bulletin and Review
DOIs
StateAccepted/In press - Nov 6 2015

Fingerprint

Stimulus
Inference

Keywords

  • Coefficient alpha
  • Generalized coefficient alpha
  • Reliability
  • Split-half reliability

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Arts and Humanities (miscellaneous)
  • Developmental and Educational Psychology

Cite this

Use of internal consistency coefficients for estimating reliability of experimental task scores. / Green, Samuel B.; Yang, Yanyun; Alt, Mary; Brinkley, Shara; Gray, Shelley; Hogan, Tiffany; Cowan, Nelson.

In: Psychonomic Bulletin and Review, 06.11.2015.

Research output: Contribution to journalArticle

Green, Samuel B. ; Yang, Yanyun ; Alt, Mary ; Brinkley, Shara ; Gray, Shelley ; Hogan, Tiffany ; Cowan, Nelson. / Use of internal consistency coefficients for estimating reliability of experimental task scores. In: Psychonomic Bulletin and Review. 2015.
@article{2a0e7c4ebcf347caa27fef828d05bb2b,
title = "Use of internal consistency coefficients for estimating reliability of experimental task scores",
abstract = "Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks.",
keywords = "Coefficient alpha, Generalized coefficient alpha, Reliability, Split-half reliability",
author = "Green, {Samuel B.} and Yanyun Yang and Mary Alt and Shara Brinkley and Shelley Gray and Tiffany Hogan and Nelson Cowan",
year = "2015",
month = "11",
day = "6",
doi = "10.3758/s13423-015-0968-3",
language = "English (US)",
journal = "Psychonomic Bulletin and Review",
issn = "1069-9384",
publisher = "Springer New York",

}

TY - JOUR

T1 - Use of internal consistency coefficients for estimating reliability of experimental task scores

AU - Green, Samuel B.

AU - Yang, Yanyun

AU - Alt, Mary

AU - Brinkley, Shara

AU - Gray, Shelley

AU - Hogan, Tiffany

AU - Cowan, Nelson

PY - 2015/11/6

Y1 - 2015/11/6

N2 - Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks.

AB - Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks.

KW - Coefficient alpha

KW - Generalized coefficient alpha

KW - Reliability

KW - Split-half reliability

UR - http://www.scopus.com/inward/record.url?scp=84946763369&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84946763369&partnerID=8YFLogxK

U2 - 10.3758/s13423-015-0968-3

DO - 10.3758/s13423-015-0968-3

M3 - Article

JO - Psychonomic Bulletin and Review

JF - Psychonomic Bulletin and Review

SN - 1069-9384

ER -