Minimalist plans for interpreting manipulation actions

Anupam Guha, Yezhou Yang, Cornelia Fermuuller, Yiannis Aloimonos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

Humans attribute meaning to actions, and can recognize, imitate, predict, compose from parts, and analyse complex actions performed by other humans. We have built a model of action representation and understanding which takes as input perceptual data of humans performing manipulatory actions and finds a semantic interpretation of it. It achieves this by representing actions as minimal plans based on a few primitives. The motivation for our approach is to have a description, that abstracts away the variations in the way humans perform actions. The model can be used to represent complex activities on the basis of simple actions. The primitives of these minimal plans are embodied in the physicality of the system doing the analysis. The model understands an action under observation by recognising which plan is occurring. Using primitives thus rooted in its own physical structure, the model has a semanticist and causal understanding of what it observes. Using plans, the model considers actions as well as complex activities in terms of causality, compositions, and goal achievement, enabling it to perform complex tasks like prediction of primitives, separation of interleaved actions and filtering of perceptual input. We use our model over an action dataset involving humans using hand tools on objects in a constrained universe to understand an activity it has not seen before in terms of actions whose plans it knows of. The model thus illustrates a novel approach of understanding human actions by a robot.

Original languageEnglish (US)
Title of host publicationIROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems
Pages5908-5914
Number of pages7
DOIs
StatePublished - 2013
Externally publishedYes
Event2013 26th IEEE/RSJ International Conference on Intelligent Robots and Systems: New Horizon, IROS 2013 - Tokyo, Japan
Duration: Nov 3 2013Nov 8 2013

Other

Other2013 26th IEEE/RSJ International Conference on Intelligent Robots and Systems: New Horizon, IROS 2013
CountryJapan
CityTokyo
Period11/3/1311/8/13

Fingerprint

Hand tools
Semantics
Robots
Chemical analysis

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Cite this

Guha, A., Yang, Y., Fermuuller, C., & Aloimonos, Y. (2013). Minimalist plans for interpreting manipulation actions. In IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 5908-5914). [6697213] https://doi.org/10.1109/IROS.2013.6697213

Minimalist plans for interpreting manipulation actions. / Guha, Anupam; Yang, Yezhou; Fermuuller, Cornelia; Aloimonos, Yiannis.

IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2013. p. 5908-5914 6697213.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Guha, A, Yang, Y, Fermuuller, C & Aloimonos, Y 2013, Minimalist plans for interpreting manipulation actions. in IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems., 6697213, pp. 5908-5914, 2013 26th IEEE/RSJ International Conference on Intelligent Robots and Systems: New Horizon, IROS 2013, Tokyo, Japan, 11/3/13. https://doi.org/10.1109/IROS.2013.6697213
Guha A, Yang Y, Fermuuller C, Aloimonos Y. Minimalist plans for interpreting manipulation actions. In IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2013. p. 5908-5914. 6697213 https://doi.org/10.1109/IROS.2013.6697213
Guha, Anupam ; Yang, Yezhou ; Fermuuller, Cornelia ; Aloimonos, Yiannis. / Minimalist plans for interpreting manipulation actions. IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2013. pp. 5908-5914
@inproceedings{9aafef9c53ca48f183c3ffb1e10e1bf0,
title = "Minimalist plans for interpreting manipulation actions",
abstract = "Humans attribute meaning to actions, and can recognize, imitate, predict, compose from parts, and analyse complex actions performed by other humans. We have built a model of action representation and understanding which takes as input perceptual data of humans performing manipulatory actions and finds a semantic interpretation of it. It achieves this by representing actions as minimal plans based on a few primitives. The motivation for our approach is to have a description, that abstracts away the variations in the way humans perform actions. The model can be used to represent complex activities on the basis of simple actions. The primitives of these minimal plans are embodied in the physicality of the system doing the analysis. The model understands an action under observation by recognising which plan is occurring. Using primitives thus rooted in its own physical structure, the model has a semanticist and causal understanding of what it observes. Using plans, the model considers actions as well as complex activities in terms of causality, compositions, and goal achievement, enabling it to perform complex tasks like prediction of primitives, separation of interleaved actions and filtering of perceptual input. We use our model over an action dataset involving humans using hand tools on objects in a constrained universe to understand an activity it has not seen before in terms of actions whose plans it knows of. The model thus illustrates a novel approach of understanding human actions by a robot.",
author = "Anupam Guha and Yezhou Yang and Cornelia Fermuuller and Yiannis Aloimonos",
year = "2013",
doi = "10.1109/IROS.2013.6697213",
language = "English (US)",
isbn = "9781467363587",
pages = "5908--5914",
booktitle = "IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems",

}

TY - GEN

T1 - Minimalist plans for interpreting manipulation actions

AU - Guha, Anupam

AU - Yang, Yezhou

AU - Fermuuller, Cornelia

AU - Aloimonos, Yiannis

PY - 2013

Y1 - 2013

N2 - Humans attribute meaning to actions, and can recognize, imitate, predict, compose from parts, and analyse complex actions performed by other humans. We have built a model of action representation and understanding which takes as input perceptual data of humans performing manipulatory actions and finds a semantic interpretation of it. It achieves this by representing actions as minimal plans based on a few primitives. The motivation for our approach is to have a description, that abstracts away the variations in the way humans perform actions. The model can be used to represent complex activities on the basis of simple actions. The primitives of these minimal plans are embodied in the physicality of the system doing the analysis. The model understands an action under observation by recognising which plan is occurring. Using primitives thus rooted in its own physical structure, the model has a semanticist and causal understanding of what it observes. Using plans, the model considers actions as well as complex activities in terms of causality, compositions, and goal achievement, enabling it to perform complex tasks like prediction of primitives, separation of interleaved actions and filtering of perceptual input. We use our model over an action dataset involving humans using hand tools on objects in a constrained universe to understand an activity it has not seen before in terms of actions whose plans it knows of. The model thus illustrates a novel approach of understanding human actions by a robot.

AB - Humans attribute meaning to actions, and can recognize, imitate, predict, compose from parts, and analyse complex actions performed by other humans. We have built a model of action representation and understanding which takes as input perceptual data of humans performing manipulatory actions and finds a semantic interpretation of it. It achieves this by representing actions as minimal plans based on a few primitives. The motivation for our approach is to have a description, that abstracts away the variations in the way humans perform actions. The model can be used to represent complex activities on the basis of simple actions. The primitives of these minimal plans are embodied in the physicality of the system doing the analysis. The model understands an action under observation by recognising which plan is occurring. Using primitives thus rooted in its own physical structure, the model has a semanticist and causal understanding of what it observes. Using plans, the model considers actions as well as complex activities in terms of causality, compositions, and goal achievement, enabling it to perform complex tasks like prediction of primitives, separation of interleaved actions and filtering of perceptual input. We use our model over an action dataset involving humans using hand tools on objects in a constrained universe to understand an activity it has not seen before in terms of actions whose plans it knows of. The model thus illustrates a novel approach of understanding human actions by a robot.

UR - http://www.scopus.com/inward/record.url?scp=84893751243&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893751243&partnerID=8YFLogxK

U2 - 10.1109/IROS.2013.6697213

DO - 10.1109/IROS.2013.6697213

M3 - Conference contribution

AN - SCOPUS:84893751243

SN - 9781467363587

SP - 5908

EP - 5914

BT - IROS 2013: New Horizon, Conference Digest - 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems

ER -