The appearance of coherence: Using cohesive properties of readers’ constructed responses to predict individual differences

Successful text comprehension requires readers to engage in a number of coherence-building processes. This study examined how analyzing the cohesion of students’ constructed responses can be used to evaluate these coherence-building processes and the extent to which they vary across readers’ individual differences and across types of texts. We posed two primary research questions: 1) Can we predict individual differences in working memory and reading skill based on the cohesion of students’ constructed responses to text? 2) Do the relations between individual differences and cohesion vary as a function of genre? Participants (n = 119) generated constructed responses while reading history and science texts and completed reading skill and working memory assessments. The current study leveraged natural language processing (NLP) techniques to analyze the cohesion of readers’ constructed responses, using cohesion as a proxy for assessing the coherence of their mental representations of the texts. Cohesion was measured at the sentence, paragraph, and synonym levels. Machine learning models showed that linguistic indices related to cohesion were significant predictors of both working memory and reading skill. Additional quantitative and qualitative inspection revealed that the relations between individual differences and coherence-building processes varied depending on the text’s genre. These findings indicate that the interaction between genre and individual differences may be used to model coherence-building processes during reading. This study has important implications for the realm of educational technology such as in the implementation of stealth assessments to predict students’ cognitive abilities.