TY - JOUR
T1 - Modeling student evaluations of writing and authors as a function of writing errors
AU - Roscoe, Rod
AU - Wilson, Joshua
AU - Patchan, Melissa
AU - Chen, Dandan
AU - Johnson, Adam
N1 - Funding Information:
This research was supported in part by a Fulton Undergraduate Research Initiative award to Adam C. Johnson from the Ira A. Fulton Schools of Engineering at Arizona State University.
Publisher Copyright:
© 2020 National Research University Higher School of Economics. All rights reserved.
PY - 2020/6/30
Y1 - 2020/6/30
N2 - Writers are often judged by their audience, and these evaluations can encompass both the text and the authors. This study built upon prior research on writing evaluation and error perceptions to examine how interconnected or separable are these judgments. Using a withinsubjects design, college students evaluated four essays demonstrating no errors, lower-level errors, higher-level errors, or both types. Evaluations included writing quality traits (e.g., conventions, ideas, organization, sentence fluency, and voice) and author characteristics (e.g., creativity, intelligence, generosity, and kindness). Exploratory factor analyses identified latent constructs within these ratings. One construct, Writing Quality and Skill, appeared to combine writing traits and authors' intellectual ability (e.g., intelligence and knowledgeability). The second construct, Author Personality, seemed to comprise interpersonal author traits (e.g., kindness and loyalty). The two constructs were significantly and positively correlated. These results suggest that students tended to form holistic impressions of writing quality and authors rather than distinct judgments about individual traits. The spillover onto perceptions of authors' personal characteristics may be representative of latent biases. Student raters were also more sensitive to lower-level errors than higher-level errors. Implications for biases and training related to peer assessment are discussed.
AB - Writers are often judged by their audience, and these evaluations can encompass both the text and the authors. This study built upon prior research on writing evaluation and error perceptions to examine how interconnected or separable are these judgments. Using a withinsubjects design, college students evaluated four essays demonstrating no errors, lower-level errors, higher-level errors, or both types. Evaluations included writing quality traits (e.g., conventions, ideas, organization, sentence fluency, and voice) and author characteristics (e.g., creativity, intelligence, generosity, and kindness). Exploratory factor analyses identified latent constructs within these ratings. One construct, Writing Quality and Skill, appeared to combine writing traits and authors' intellectual ability (e.g., intelligence and knowledgeability). The second construct, Author Personality, seemed to comprise interpersonal author traits (e.g., kindness and loyalty). The two constructs were significantly and positively correlated. These results suggest that students tended to form holistic impressions of writing quality and authors rather than distinct judgments about individual traits. The spillover onto perceptions of authors' personal characteristics may be representative of latent biases. Student raters were also more sensitive to lower-level errors than higher-level errors. Implications for biases and training related to peer assessment are discussed.
KW - College students
KW - Factor analysis
KW - Peer assessment
KW - Rater bias
KW - Writing evaluation
KW - Writing instruction
UR - http://www.scopus.com/inward/record.url?scp=85090982276&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85090982276&partnerID=8YFLogxK
U2 - 10.17323/JLE.2020.10316
DO - 10.17323/JLE.2020.10316
M3 - Article
AN - SCOPUS:85090982276
SN - 2411-7390
VL - 6
SP - 147
EP - 164
JO - Journal of Language and Education
JF - Journal of Language and Education
IS - 2
ER -