Comparing machine learning classification approaches for predicting expository text difficulty

Renu Balyan, Kathryn S. McCarthy, Danielle S. McNamara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

While hierarchical machine learning approaches have been used to classify texts into different content areas, this approach has, to our knowledge, not been used in the automated assessment of text difficulty. This study compared the accuracy of four classification machine learning approaches (flat, one-vs-one, one-vs-all, and hierarchical) using natural language processing features in predicting human ratings of text difficulty for two sets of texts. The hierarchical classification was the most accurate for the two text sets considered individually (Set A, 77.78%; Set B, 82.05%), while the non-hierarchical approaches, one-vs-one and one-vs-all, performed similar to the hierarchical classification for the combined set (71.43%). These findings suggest both promise and limitations for applying hierarchical approaches to text difficulty classification. It may be beneficial to apply a recursive top-down approach to discriminate the subsets of classes that are at the top of the hierarchy and less related, and then further separate the classes into subsets that may be more similar to one other. These results also suggest that a single approach may not always work for all types of da-taseis and that it is important to evaluate which machine learning approach and algorithm works best for particular datasets. The authors encourage more work in this area to help suggest which types of algorithms work best as a function of the type of dataset.

Original languageEnglish (US)
Title of host publicationProceedings of the 31st International Florida Artificial Intelligence Research Society Conference, FLAIRS 2018
EditorsKeith Brawner, Vasile Rus
PublisherAAAI press
Pages421-426
Number of pages6
ISBN (Electronic)9781577357964
StatePublished - Jan 1 2018
Event31st International Florida Artificial Intelligence Research Society Conference, FLAIRS 2018 - Melbourne, United States
Duration: May 21 2018May 23 2018

Publication series

NameProceedings of the 31st International Florida Artificial Intelligence Research Society Conference, FLAIRS 2018

Conference

Conference31st International Florida Artificial Intelligence Research Society Conference, FLAIRS 2018
CountryUnited States
CityMelbourne
Period5/21/185/23/18

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software

Fingerprint Dive into the research topics of 'Comparing machine learning classification approaches for predicting expository text difficulty'. Together they form a unique fingerprint.

  • Cite this

    Balyan, R., McCarthy, K. S., & McNamara, D. S. (2018). Comparing machine learning classification approaches for predicting expository text difficulty. In K. Brawner, & V. Rus (Eds.), Proceedings of the 31st International Florida Artificial Intelligence Research Society Conference, FLAIRS 2018 (pp. 421-426). (Proceedings of the 31st International Florida Artificial Intelligence Research Society Conference, FLAIRS 2018). AAAI press.