Analyzing Longitudinal Multirater Data with Changing and Stable Raters

Tobias Koch, Jana Holtmann, Michael Eid, Stephen West

Research output: Contribution to journalArticle

Abstract

One issue in analyzing longitudinal multirater data arises if raters drop-in or drop-out throughout a longitudinal study. We term this issue random rater movement (RRM), assuming that the selection of raters into the study approximates a random process and is strongly ignorable. We explain how RRM can be modeled in case of longitudinal multirater designs with (a) interchangeable raters or (b) structurally different raters. To analyze measurement designs with stable and changing interchangeable raters, we recommend using a longitudinal multilevel confirmatory factor model. To analyze measurement designs with stable and changing structurally different raters, we propose a longitudinal multigroup confirmatory factor model. The proposed model is illustrated using real data. Additionally, the performance of the models with regard to a small number of raters and a relatively small overall sample size is examined in Monte Carlo simulation studies. Future directions for analyzing rater movement over time are provided.

Original languageEnglish (US)
JournalStructural Equation Modeling
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Longitudinal Data
Factor Models
Drop out
Longitudinal Study
Small Sample Size
Random process
Monte Carlo Simulation
drop-out
Random processes
Simulation Study
longitudinal study
Term
Model
simulation
Movement
Design
Longitudinal data
performance

Keywords

  • longitudinal analysis
  • missing data
  • multirater data
  • multitrait-multimethod-multioccasion modeling

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Modeling and Simulation
  • Sociology and Political Science
  • Economics, Econometrics and Finance(all)

Cite this

Analyzing Longitudinal Multirater Data with Changing and Stable Raters. / Koch, Tobias; Holtmann, Jana; Eid, Michael; West, Stephen.

In: Structural Equation Modeling, 01.01.2019.

Research output: Contribution to journalArticle

@article{7d43ed37ebbb43a9bf42ddbe03a060ea,
title = "Analyzing Longitudinal Multirater Data with Changing and Stable Raters",
abstract = "One issue in analyzing longitudinal multirater data arises if raters drop-in or drop-out throughout a longitudinal study. We term this issue random rater movement (RRM), assuming that the selection of raters into the study approximates a random process and is strongly ignorable. We explain how RRM can be modeled in case of longitudinal multirater designs with (a) interchangeable raters or (b) structurally different raters. To analyze measurement designs with stable and changing interchangeable raters, we recommend using a longitudinal multilevel confirmatory factor model. To analyze measurement designs with stable and changing structurally different raters, we propose a longitudinal multigroup confirmatory factor model. The proposed model is illustrated using real data. Additionally, the performance of the models with regard to a small number of raters and a relatively small overall sample size is examined in Monte Carlo simulation studies. Future directions for analyzing rater movement over time are provided.",
keywords = "longitudinal analysis, missing data, multirater data, multitrait-multimethod-multioccasion modeling",
author = "Tobias Koch and Jana Holtmann and Michael Eid and Stephen West",
year = "2019",
month = "1",
day = "1",
doi = "10.1080/10705511.2019.1638784",
language = "English (US)",
journal = "Structural Equation Modeling",
issn = "1070-5511",
publisher = "Psychology Press Ltd",

}

TY - JOUR

T1 - Analyzing Longitudinal Multirater Data with Changing and Stable Raters

AU - Koch, Tobias

AU - Holtmann, Jana

AU - Eid, Michael

AU - West, Stephen

PY - 2019/1/1

Y1 - 2019/1/1

N2 - One issue in analyzing longitudinal multirater data arises if raters drop-in or drop-out throughout a longitudinal study. We term this issue random rater movement (RRM), assuming that the selection of raters into the study approximates a random process and is strongly ignorable. We explain how RRM can be modeled in case of longitudinal multirater designs with (a) interchangeable raters or (b) structurally different raters. To analyze measurement designs with stable and changing interchangeable raters, we recommend using a longitudinal multilevel confirmatory factor model. To analyze measurement designs with stable and changing structurally different raters, we propose a longitudinal multigroup confirmatory factor model. The proposed model is illustrated using real data. Additionally, the performance of the models with regard to a small number of raters and a relatively small overall sample size is examined in Monte Carlo simulation studies. Future directions for analyzing rater movement over time are provided.

AB - One issue in analyzing longitudinal multirater data arises if raters drop-in or drop-out throughout a longitudinal study. We term this issue random rater movement (RRM), assuming that the selection of raters into the study approximates a random process and is strongly ignorable. We explain how RRM can be modeled in case of longitudinal multirater designs with (a) interchangeable raters or (b) structurally different raters. To analyze measurement designs with stable and changing interchangeable raters, we recommend using a longitudinal multilevel confirmatory factor model. To analyze measurement designs with stable and changing structurally different raters, we propose a longitudinal multigroup confirmatory factor model. The proposed model is illustrated using real data. Additionally, the performance of the models with regard to a small number of raters and a relatively small overall sample size is examined in Monte Carlo simulation studies. Future directions for analyzing rater movement over time are provided.

KW - longitudinal analysis

KW - missing data

KW - multirater data

KW - multitrait-multimethod-multioccasion modeling

UR - http://www.scopus.com/inward/record.url?scp=85070952076&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070952076&partnerID=8YFLogxK

U2 - 10.1080/10705511.2019.1638784

DO - 10.1080/10705511.2019.1638784

M3 - Article

JO - Structural Equation Modeling

JF - Structural Equation Modeling

SN - 1070-5511

ER -