Using a minimal action grammar for activity understanding in the real world

Douglas Summers-Stay, Ching L. Teo, Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

23 Scopus citations

Abstract

There is good reason to believe that humans use some kind of recursive grammatical structure when we recognize and perform complex manipulation activities. We have built a system to automatically build a tree structure from observations of an actor performing such activities. The activity trees that result form a framework for search and understanding, tying action to language. We explore and evaluate the system by performing experiments over a novel complex activity dataset taken using synchronized Kinect and SR4000 Time of Flight cameras. Processing of the combined 3D and 2D image data provides the necessary terminals and events to build the tree from the bottom-up. Experimental results highlight the contribution of the action grammar in: 1) providing a robust structure for complex activity recognition over real data and 2) disambiguating interleaved activities from within the same sequence.

Original languageEnglish (US)
Title of host publication2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012
Pages4104-4111
Number of pages8
DOIs
StatePublished - Dec 1 2012
Externally publishedYes
Event25th IEEE/RSJ International Conference on Robotics and Intelligent Systems, IROS 2012 - Vilamoura, Algarve, Portugal
Duration: Oct 7 2012Oct 12 2012

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Other

Other25th IEEE/RSJ International Conference on Robotics and Intelligent Systems, IROS 2012
CountryPortugal
CityVilamoura, Algarve
Period10/7/1210/12/12

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Using a minimal action grammar for activity understanding in the real world'. Together they form a unique fingerprint.

  • Cite this

    Summers-Stay, D., Teo, C. L., Yang, Y., Fermuller, C., & Aloimonos, Y. (2012). Using a minimal action grammar for activity understanding in the real world. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012 (pp. 4104-4111). [6385483] (IEEE International Conference on Intelligent Robots and Systems). https://doi.org/10.1109/IROS.2012.6385483