BAPO: A Large-Scale Multimodal Corpus for Ball Possession Prediction in American Football Games

Ziruo Yi, Eduardo Blanco, Heng Fan, Mark V. Albert

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a new task for multimodal information extraction: identify the players or teams that possess the ball in each play of an American football game. We also introduce BAPO, a large-scale corpus that consists of 100 games totaling around 200 hours of video broadcasts along with ball possession information for 15,132 plays. This corpus is rich and diverse because it involves a great number of players and different scenarios. BAPO poses a new challenge to build multimodal models that take into account language (what the broadcasters say), audio (broadcasters' tone, sounds from the audience, etc.) and vision (frames from the video broadcast). We further propose a baseline model, BAPOTer, and conduct comprehensive experiments. Results demonstrate that language is key to solve this task and that leveraging all three modalities is beneficial. The corpus is available at https://github.com/Isabella1118/BAPO.

Original languageEnglish (US)
Title of host publicationProceedings - 5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages391-394
Number of pages4
ISBN (Electronic)9781665495486
DOIs
StatePublished - 2022
Event5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022 - Virtual, Online, United States
Duration: Aug 2 2022Aug 4 2022

Publication series

NameProceedings - 5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022

Conference

Conference5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022
Country/TerritoryUnited States
CityVirtual, Online
Period8/2/228/4/22

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Signal Processing
  • Safety, Risk, Reliability and Quality
  • Media Technology

Fingerprint

Dive into the research topics of 'BAPO: A Large-Scale Multimodal Corpus for Ball Possession Prediction in American Football Games'. Together they form a unique fingerprint.

Cite this