Using multimodal data for automated fidelity evaluation in pivotal response treatment videos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Research has shown that caregivers implementing pivotal response treatment (PRT) with their child with autism spectrum disorder (ASD) helps the child develop social and communication skills. Evaluation of caregiver fidelity to PRT in training programs and research studies relies on the evaluation of video probes depicting the caregiver interacting with his or her child. These video probes are reviewed by behavior analysts and are dependent on manual processing to extract data metrics. Using multimodal data processing techniques and machine learning could alleviate the human cost of evaluating the video probes by automating data analysis tasks.Creating an 'Opportunity to Respond' is one of the categories used to evaluate caregiver fidelity to PRT implementation. A caregiver is determined to have successfully demonstrated cre-ating an opportunity to respond when they have delivered an appropriate instruction while she or he has the child's attention. Automatically determining when the caregiver has correctly provided an opportunity to respond requires classifying the audio and video data from the probes. Combining the modalities into a single classification task can be undertaken using feature fusion or decision fusion methods. Two decision fusion configurations, and a feature fusion model were evaluated. The decision fusion models achieved higher accuracy, however the feature fusion model had a higher average F1 score, indicating more reliable prediction capability.

Original languageEnglish (US)
Title of host publicationGlobalSIP 2019 - 7th IEEE Global Conference on Signal and Information Processing, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728127231
DOIs
StatePublished - Nov 2019
Event7th IEEE Global Conference on Signal and Information Processing, GlobalSIP 2019 - Ottawa, Canada
Duration: Nov 11 2019Nov 14 2019

Publication series

NameGlobalSIP 2019 - 7th IEEE Global Conference on Signal and Information Processing, Proceedings

Conference

Conference7th IEEE Global Conference on Signal and Information Processing, GlobalSIP 2019
CountryCanada
CityOttawa
Period11/11/1911/14/19

Keywords

  • Attention Detection
  • Autism Spectrum Disorder
  • Machine Learning
  • Multimodal Data
  • Pivotal Response Treatment

ASJC Scopus subject areas

  • Information Systems
  • Information Systems and Management
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint Dive into the research topics of 'Using multimodal data for automated fidelity evaluation in pivotal response treatment videos'. Together they form a unique fingerprint.

  • Cite this

    Heath, C. D. C., Venkateswara, H., McDaniel, T., & Panchanathan, S. (2019). Using multimodal data for automated fidelity evaluation in pivotal response treatment videos. In GlobalSIP 2019 - 7th IEEE Global Conference on Signal and Information Processing, Proceedings [8969089] (GlobalSIP 2019 - 7th IEEE Global Conference on Signal and Information Processing, Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/GlobalSIP45357.2019.8969089