Large-scale analysis of test–retest reliabilities of self-regulation measures

A. Zeynep Enkavi, Ian W. Eisenberg, Patrick G. Bissett, Gina L. Mazza, David Mackinnon, Lisa A. Marsch, Russell A. Poldrack

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

The ability to regulate behavior in service of long-term goals is a widely studied psychological construct known as self-regulation. This wide interest is in part due to the putative relations between self-regulation and a range of real-world behaviors. Self-regulation is generally viewed as a trait, and individual differences are quantified using a diverse set of measures, including self-report surveys and behavioral tasks. Accurate characterization of individual differences requires measurement reliability, a property frequently characterized in self-report surveys, but rarely assessed in behavioral tasks. We remedy this gap by (i) providing a comprehensive literature review on an extensive set of self-regulation measures and (ii) empirically evaluating test–retest reliability of this battery in a new sample. We find that dependent variables (DVs) from self-report surveys of self-regulation have high test–retest reliability, while DVs derived from behavioral tasks do not. This holds both in the literature and in our sample, although the test–retest reliability estimates in the literature are highly variable. We confirm that this is due to differences in between-subject variability. We also compare different types of task DVs (e.g., model parameters vs. raw response times) in their suitability as individual difference DVs, finding that certain model parameters are as stable as raw DVs. Our results provide greater psychometric footing for the study of self-regulation and provide guidance for future studies of individual differences in this domain.

Original languageEnglish (US)
Pages (from-to)5472-5477
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume116
Issue number12
DOIs
StatePublished - Jan 1 2019

Fingerprint

Individuality
Self Report
Aptitude
Psychometrics
Reaction Time
Self-Control
Psychology
Surveys and Questionnaires

Keywords

  • Individual differences
  • Retest reliability
  • Self-regulation

ASJC Scopus subject areas

  • General

Cite this

Large-scale analysis of test–retest reliabilities of self-regulation measures. / Zeynep Enkavi, A.; Eisenberg, Ian W.; Bissett, Patrick G.; Mazza, Gina L.; Mackinnon, David; Marsch, Lisa A.; Poldrack, Russell A.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 116, No. 12, 01.01.2019, p. 5472-5477.

Research output: Contribution to journalArticle

Zeynep Enkavi, A. ; Eisenberg, Ian W. ; Bissett, Patrick G. ; Mazza, Gina L. ; Mackinnon, David ; Marsch, Lisa A. ; Poldrack, Russell A. / Large-scale analysis of test–retest reliabilities of self-regulation measures. In: Proceedings of the National Academy of Sciences of the United States of America. 2019 ; Vol. 116, No. 12. pp. 5472-5477.
@article{4963df3ab9d24244b8747294b507bcfe,
title = "Large-scale analysis of test–retest reliabilities of self-regulation measures",
abstract = "The ability to regulate behavior in service of long-term goals is a widely studied psychological construct known as self-regulation. This wide interest is in part due to the putative relations between self-regulation and a range of real-world behaviors. Self-regulation is generally viewed as a trait, and individual differences are quantified using a diverse set of measures, including self-report surveys and behavioral tasks. Accurate characterization of individual differences requires measurement reliability, a property frequently characterized in self-report surveys, but rarely assessed in behavioral tasks. We remedy this gap by (i) providing a comprehensive literature review on an extensive set of self-regulation measures and (ii) empirically evaluating test–retest reliability of this battery in a new sample. We find that dependent variables (DVs) from self-report surveys of self-regulation have high test–retest reliability, while DVs derived from behavioral tasks do not. This holds both in the literature and in our sample, although the test–retest reliability estimates in the literature are highly variable. We confirm that this is due to differences in between-subject variability. We also compare different types of task DVs (e.g., model parameters vs. raw response times) in their suitability as individual difference DVs, finding that certain model parameters are as stable as raw DVs. Our results provide greater psychometric footing for the study of self-regulation and provide guidance for future studies of individual differences in this domain.",
keywords = "Individual differences, Retest reliability, Self-regulation",
author = "{Zeynep Enkavi}, A. and Eisenberg, {Ian W.} and Bissett, {Patrick G.} and Mazza, {Gina L.} and David Mackinnon and Marsch, {Lisa A.} and Poldrack, {Russell A.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1073/pnas.1818430116",
language = "English (US)",
volume = "116",
pages = "5472--5477",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
number = "12",

}

TY - JOUR

T1 - Large-scale analysis of test–retest reliabilities of self-regulation measures

AU - Zeynep Enkavi, A.

AU - Eisenberg, Ian W.

AU - Bissett, Patrick G.

AU - Mazza, Gina L.

AU - Mackinnon, David

AU - Marsch, Lisa A.

AU - Poldrack, Russell A.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The ability to regulate behavior in service of long-term goals is a widely studied psychological construct known as self-regulation. This wide interest is in part due to the putative relations between self-regulation and a range of real-world behaviors. Self-regulation is generally viewed as a trait, and individual differences are quantified using a diverse set of measures, including self-report surveys and behavioral tasks. Accurate characterization of individual differences requires measurement reliability, a property frequently characterized in self-report surveys, but rarely assessed in behavioral tasks. We remedy this gap by (i) providing a comprehensive literature review on an extensive set of self-regulation measures and (ii) empirically evaluating test–retest reliability of this battery in a new sample. We find that dependent variables (DVs) from self-report surveys of self-regulation have high test–retest reliability, while DVs derived from behavioral tasks do not. This holds both in the literature and in our sample, although the test–retest reliability estimates in the literature are highly variable. We confirm that this is due to differences in between-subject variability. We also compare different types of task DVs (e.g., model parameters vs. raw response times) in their suitability as individual difference DVs, finding that certain model parameters are as stable as raw DVs. Our results provide greater psychometric footing for the study of self-regulation and provide guidance for future studies of individual differences in this domain.

AB - The ability to regulate behavior in service of long-term goals is a widely studied psychological construct known as self-regulation. This wide interest is in part due to the putative relations between self-regulation and a range of real-world behaviors. Self-regulation is generally viewed as a trait, and individual differences are quantified using a diverse set of measures, including self-report surveys and behavioral tasks. Accurate characterization of individual differences requires measurement reliability, a property frequently characterized in self-report surveys, but rarely assessed in behavioral tasks. We remedy this gap by (i) providing a comprehensive literature review on an extensive set of self-regulation measures and (ii) empirically evaluating test–retest reliability of this battery in a new sample. We find that dependent variables (DVs) from self-report surveys of self-regulation have high test–retest reliability, while DVs derived from behavioral tasks do not. This holds both in the literature and in our sample, although the test–retest reliability estimates in the literature are highly variable. We confirm that this is due to differences in between-subject variability. We also compare different types of task DVs (e.g., model parameters vs. raw response times) in their suitability as individual difference DVs, finding that certain model parameters are as stable as raw DVs. Our results provide greater psychometric footing for the study of self-regulation and provide guidance for future studies of individual differences in this domain.

KW - Individual differences

KW - Retest reliability

KW - Self-regulation

UR - http://www.scopus.com/inward/record.url?scp=85063266523&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063266523&partnerID=8YFLogxK

U2 - 10.1073/pnas.1818430116

DO - 10.1073/pnas.1818430116

M3 - Article

VL - 116

SP - 5472

EP - 5477

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 12

ER -