TY - JOUR
T1 - Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL
AU - Cumming, Alister
AU - Kantor, Robert
AU - Baba, Kyoko
AU - Erdosy, Usman
AU - Eouanzoui, Keanre
AU - James, Mark
N1 - Funding Information:
This research was funded by the TOEFL research program at Educational Testing Service. The opinions expressed are those of the authors rather than the policies of the TOEFL or of Educational Testing Service. An extended version of this article will appear as a TOEFL Research Report. We thank Dan Eignor for helpful suggestions on our reporting of statistics as well as Don Banh for designing the computer software program for analyses of verbatim strings of words from source texts, type-token ratios of lexical words, and word length.
PY - 2005
Y1 - 2005
N2 - We assessed whether and how the discourse written for prototype integrated tasks (involving writing in response to print or audio source texts) field tested for Next Generation TOEFL® differs from the discourse written for independent essays (i.e., the TOEFL Essay®. We selected 216 compositions written for six tasks by 36 examinees in a field test-representing score levels 3, 4, and 5 on the TOEFL Essay-then coded the texts for lexical and syntactic complexity, grammatical accuracy, argument structure, orientations to evidence, and verbatim uses of source text. Analyses with non-parametric MANOVAs followed a three (task type: TOEFL Essay, writing in response to a reading passage, writing in response to a listening passage) by three (English proficiency level: score levels 3, 4, and 5 on the TOEFL Essay) within-subjects factorial design. The discourse produced for the integrated writing tasks differed significantly from the discourse produced in the independent essay for the variables of: lexical complexity (text length, word length, ratio of different words to total words written), syntactic complexity (number of words per T-unit, number clauses per T-unit), rhetoric (quality of propositions, claims, data, warrants, and oppositions in argument structure), and pragmatics (orientations to source evidence in respect to self or others and to phrasing the message as either declarations, paraphrases, or summaries). Across the three English proficiency levels, significant differences appeared for the variables of grammatical accuracy as well as all indicators of lexical complexity (text length, word length, ratio of different words to total words written), one indicator of syntactic complexity (words per T-unit), one rhetorical aspect (quality of claims in argument structure), and two pragmatic aspects (expression of self as voice, messages phrased as summaries).
AB - We assessed whether and how the discourse written for prototype integrated tasks (involving writing in response to print or audio source texts) field tested for Next Generation TOEFL® differs from the discourse written for independent essays (i.e., the TOEFL Essay®. We selected 216 compositions written for six tasks by 36 examinees in a field test-representing score levels 3, 4, and 5 on the TOEFL Essay-then coded the texts for lexical and syntactic complexity, grammatical accuracy, argument structure, orientations to evidence, and verbatim uses of source text. Analyses with non-parametric MANOVAs followed a three (task type: TOEFL Essay, writing in response to a reading passage, writing in response to a listening passage) by three (English proficiency level: score levels 3, 4, and 5 on the TOEFL Essay) within-subjects factorial design. The discourse produced for the integrated writing tasks differed significantly from the discourse produced in the independent essay for the variables of: lexical complexity (text length, word length, ratio of different words to total words written), syntactic complexity (number of words per T-unit, number clauses per T-unit), rhetoric (quality of propositions, claims, data, warrants, and oppositions in argument structure), and pragmatics (orientations to source evidence in respect to self or others and to phrasing the message as either declarations, paraphrases, or summaries). Across the three English proficiency levels, significant differences appeared for the variables of grammatical accuracy as well as all indicators of lexical complexity (text length, word length, ratio of different words to total words written), one indicator of syntactic complexity (words per T-unit), one rhetorical aspect (quality of claims in argument structure), and two pragmatic aspects (expression of self as voice, messages phrased as summaries).
KW - Discourse analysis
KW - ESL writing
KW - Integrated tasks
KW - TOEFL
KW - Writing assessment
UR - http://www.scopus.com/inward/record.url?scp=18844439204&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=18844439204&partnerID=8YFLogxK
U2 - 10.1016/j.asw.2005.02.001
DO - 10.1016/j.asw.2005.02.001
M3 - Article
AN - SCOPUS:18844439204
SN - 1075-2935
VL - 10
SP - 5
EP - 43
JO - Assessing Writing
JF - Assessing Writing
IS - 1
ER -