Assessing text readability using cognitively based indices

Many programs designed to compute the readability of texts are narrowly based on surface-level linguistic features and take too little account of the processes which a reader brings to the text. This study is an exploratory examination of the use of Coh-Metrix, a computational tool that measures cohesion and text difficulty at various levels of language, discourse, and conceptual analysis. It is suggested that Coh-Metrix provides an improved means of measuring English text readability for second language (L2) readers, not least because three Coh-Metrix variables, one employing lexical coreferentiality, one measuring syntactic sentence similarity, and one measuring word frequency, have correlates in psycholinguistic theory. The current study draws on the validation exercise conducted by Greenfield (1999) with Japanese EFL students, which partially replicated Bormuth's (1971) study with American students. It finds that Coh-Metrix, with its inclusion of the three variables, yields a more accurate prediction of reading difficulty than traditional readability measures. The finding indicates that linguistic variables related to cognitive reading processes contribute significantly to better readability prediction than the surface variables used in traditional formulas. Additionally, because these Coh-Metrix variables better reflect psycholinguistic factors in reading comprehension such as decoding, syntactic parsing, and meaning construction, the formula appears to be more soundly based and avoids criticism on the grounds of construct validity.