Are you paying attention? Classifying attention in pivotal response treatment videos

Pivotal response treatment (PRT) has been empirically shown to aid children with autism spectrum disorder ASD improve their communication skills. The child’s primary caregivers can effectively implement PRT when provided with training and support, leading to greater opportunities for the child to improve. Utilization of computer vision technology is a critical component of creating more opportunities to support individuals implementing PRT. Automatically extracting data from videos of caregivers’ interactions with their child during PRT sessions would alleviate the human effort required to provide assessment and feedback, which would allow experts to provide greater support to more individuals. Additionally, this data could be used to provide immediate automated feedback. The process of extracting data from PRT videos is complicated and provides excellent context for a computer vision challenge. PRT videos consist of’in-the-wild’ conditions of dyadic interactions recorded on ubiquitously available devices, and vary in filming quality. The challenge presented tasks researchers with inferring the child’s attention state in relation to the caregiver in the video based on body pose information and visual cues. Approaches will be evaluated based on accuracy metrics, however, the algorithm’s speed is also important. Having fast algorithms will reduce the time between performance and assessment, allowing for greater opportunities to situate feedback in the context of the learning activity. Low-power solutions are also necessary to accommodate delivering results on mobile devices.