Surveying for "artifacts"

The susceptibility of the ocb-performance evaluation relationship to common rater, item, and measurement context effects

Nathan P. Podsakoff, Steven W. Whiting, David Welsh, Ke Michael Mai

Research output: Contribution to journalArticle

29 Citations (Scopus)

Abstract

Despite the increased attention paid to biases attributable to common method variance (CMV) over the past 50 years, researchers have only recently begun to systematically examine the effect of specific sources of CMV in previously published empirical studies. Our study contributes to this research by examining the extent to which common rater, item, and measurement context characteristics bias the relationships between organizational citizenship behaviors and performance evaluations using a mixedeffects analytic technique. Results from 173 correlations reported in 81 empirical studies (N = 31,146) indicate that even after controlling for study-level factors, common rater and anchor point number similarity substantially biased the focal correlations. Indeed, these sources of CMV (a) led to estimates that were between 60% and 96% larger when comparing measures obtained from a common rater, versus different raters; (b) led to 39% larger estimates when a common source rated the scales using the same number, versus a different number, of anchor points; and (c) when taken together with other study-level predictors, accounted for over half of the between-study variance in the focal correlations. We discuss the implications for researchers and practitioners and provide recommendations for future research.

Original languageEnglish (US)
Pages (from-to)863-874
Number of pages12
JournalJournal of Applied Psychology
Volume98
Issue number5
DOIs
StatePublished - Sep 2013
Externally publishedYes

Fingerprint

Artifacts
Research Personnel
Research

Keywords

  • Common method variance bias
  • Employee performance evaluations
  • Organizational citizenship behaviors
  • Rating source
  • Scale anchor points

ASJC Scopus subject areas

  • Applied Psychology

Cite this

Surveying for "artifacts" : The susceptibility of the ocb-performance evaluation relationship to common rater, item, and measurement context effects. / Podsakoff, Nathan P.; Whiting, Steven W.; Welsh, David; Mai, Ke Michael.

In: Journal of Applied Psychology, Vol. 98, No. 5, 09.2013, p. 863-874.

Research output: Contribution to journalArticle

@article{e525ab865fd1445a866eecfc285f23f5,
title = "Surveying for {"}artifacts{"}: The susceptibility of the ocb-performance evaluation relationship to common rater, item, and measurement context effects",
abstract = "Despite the increased attention paid to biases attributable to common method variance (CMV) over the past 50 years, researchers have only recently begun to systematically examine the effect of specific sources of CMV in previously published empirical studies. Our study contributes to this research by examining the extent to which common rater, item, and measurement context characteristics bias the relationships between organizational citizenship behaviors and performance evaluations using a mixedeffects analytic technique. Results from 173 correlations reported in 81 empirical studies (N = 31,146) indicate that even after controlling for study-level factors, common rater and anchor point number similarity substantially biased the focal correlations. Indeed, these sources of CMV (a) led to estimates that were between 60{\%} and 96{\%} larger when comparing measures obtained from a common rater, versus different raters; (b) led to 39{\%} larger estimates when a common source rated the scales using the same number, versus a different number, of anchor points; and (c) when taken together with other study-level predictors, accounted for over half of the between-study variance in the focal correlations. We discuss the implications for researchers and practitioners and provide recommendations for future research.",
keywords = "Common method variance bias, Employee performance evaluations, Organizational citizenship behaviors, Rating source, Scale anchor points",
author = "Podsakoff, {Nathan P.} and Whiting, {Steven W.} and David Welsh and Mai, {Ke Michael}",
year = "2013",
month = "9",
doi = "10.1037/a0032588",
language = "English (US)",
volume = "98",
pages = "863--874",
journal = "Journal of Applied Psychology",
issn = "0021-9010",
publisher = "American Psychological Association Inc.",
number = "5",

}

TY - JOUR

T1 - Surveying for "artifacts"

T2 - The susceptibility of the ocb-performance evaluation relationship to common rater, item, and measurement context effects

AU - Podsakoff, Nathan P.

AU - Whiting, Steven W.

AU - Welsh, David

AU - Mai, Ke Michael

PY - 2013/9

Y1 - 2013/9

N2 - Despite the increased attention paid to biases attributable to common method variance (CMV) over the past 50 years, researchers have only recently begun to systematically examine the effect of specific sources of CMV in previously published empirical studies. Our study contributes to this research by examining the extent to which common rater, item, and measurement context characteristics bias the relationships between organizational citizenship behaviors and performance evaluations using a mixedeffects analytic technique. Results from 173 correlations reported in 81 empirical studies (N = 31,146) indicate that even after controlling for study-level factors, common rater and anchor point number similarity substantially biased the focal correlations. Indeed, these sources of CMV (a) led to estimates that were between 60% and 96% larger when comparing measures obtained from a common rater, versus different raters; (b) led to 39% larger estimates when a common source rated the scales using the same number, versus a different number, of anchor points; and (c) when taken together with other study-level predictors, accounted for over half of the between-study variance in the focal correlations. We discuss the implications for researchers and practitioners and provide recommendations for future research.

AB - Despite the increased attention paid to biases attributable to common method variance (CMV) over the past 50 years, researchers have only recently begun to systematically examine the effect of specific sources of CMV in previously published empirical studies. Our study contributes to this research by examining the extent to which common rater, item, and measurement context characteristics bias the relationships between organizational citizenship behaviors and performance evaluations using a mixedeffects analytic technique. Results from 173 correlations reported in 81 empirical studies (N = 31,146) indicate that even after controlling for study-level factors, common rater and anchor point number similarity substantially biased the focal correlations. Indeed, these sources of CMV (a) led to estimates that were between 60% and 96% larger when comparing measures obtained from a common rater, versus different raters; (b) led to 39% larger estimates when a common source rated the scales using the same number, versus a different number, of anchor points; and (c) when taken together with other study-level predictors, accounted for over half of the between-study variance in the focal correlations. We discuss the implications for researchers and practitioners and provide recommendations for future research.

KW - Common method variance bias

KW - Employee performance evaluations

KW - Organizational citizenship behaviors

KW - Rating source

KW - Scale anchor points

UR - http://www.scopus.com/inward/record.url?scp=84884217277&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84884217277&partnerID=8YFLogxK

U2 - 10.1037/a0032588

DO - 10.1037/a0032588

M3 - Article

VL - 98

SP - 863

EP - 874

JO - Journal of Applied Psychology

JF - Journal of Applied Psychology

SN - 0021-9010

IS - 5

ER -