Detection of manipulation action consequences (MAC)

Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos

Research output: Contribution to journalArticle

45 Citations (Scopus)

Abstract

The problem of action recognition and human activity has been an active research area in Computer Vision and Robotics. While full-body motions can be characterized by movement and change of posture, no characterization, that holds invariance, has yet been proposed for the description of manipulation actions. We propose that a fundamental concept in understanding such actions, are the consequences of actions. There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions. In this paper a technique is developed to recognize these action consequences. At the heart of the technique lies a novel active tracking and segmentation method that monitors the changes in appearance and topological structure of the manipulated object. These are then used in a visual semantic graph (VSG) based procedure applied to the time sequence of the monitored object to recognize the action consequence. We provide a new dataset, called anipulation Action Consequences (MAC 1.0), which can serve as test bed for other studies on this topic. Several experiments on this dataset demonstrates that our method can robustly track objects and detect their deformations and division during the manipulation. Quantitative tests prove the effectiveness and efficiency of the method.

Original languageEnglish (US)
Article number6619175
Pages (from-to)2563-2570
Number of pages8
JournalUnknown Journal
DOIs
StatePublished - 2013
Externally publishedYes

Fingerprint

Invariance
Computer vision
Robotics
Semantics
Experiments
Posture
Human Activities
Research
Datasets

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Detection of manipulation action consequences (MAC). / Yang, Yezhou; Fermuller, Cornelia; Aloimonos, Yiannis.

In: Unknown Journal, 2013, p. 2563-2570.

Research output: Contribution to journalArticle

Yang, Yezhou ; Fermuller, Cornelia ; Aloimonos, Yiannis. / Detection of manipulation action consequences (MAC). In: Unknown Journal. 2013 ; pp. 2563-2570.
@article{d09ac6baafcb4a23b428fbb38f596825,
title = "Detection of manipulation action consequences (MAC)",
abstract = "The problem of action recognition and human activity has been an active research area in Computer Vision and Robotics. While full-body motions can be characterized by movement and change of posture, no characterization, that holds invariance, has yet been proposed for the description of manipulation actions. We propose that a fundamental concept in understanding such actions, are the consequences of actions. There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions. In this paper a technique is developed to recognize these action consequences. At the heart of the technique lies a novel active tracking and segmentation method that monitors the changes in appearance and topological structure of the manipulated object. These are then used in a visual semantic graph (VSG) based procedure applied to the time sequence of the monitored object to recognize the action consequence. We provide a new dataset, called anipulation Action Consequences (MAC 1.0), which can serve as test bed for other studies on this topic. Several experiments on this dataset demonstrates that our method can robustly track objects and detect their deformations and division during the manipulation. Quantitative tests prove the effectiveness and efficiency of the method.",
author = "Yezhou Yang and Cornelia Fermuller and Yiannis Aloimonos",
year = "2013",
doi = "10.1109/CVPR.2013.331",
language = "English (US)",
pages = "2563--2570",
journal = "Scanning Electron Microscopy",
issn = "0586-5581",
publisher = "Scanning Microscopy International",

}

TY - JOUR

T1 - Detection of manipulation action consequences (MAC)

AU - Yang, Yezhou

AU - Fermuller, Cornelia

AU - Aloimonos, Yiannis

PY - 2013

Y1 - 2013

N2 - The problem of action recognition and human activity has been an active research area in Computer Vision and Robotics. While full-body motions can be characterized by movement and change of posture, no characterization, that holds invariance, has yet been proposed for the description of manipulation actions. We propose that a fundamental concept in understanding such actions, are the consequences of actions. There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions. In this paper a technique is developed to recognize these action consequences. At the heart of the technique lies a novel active tracking and segmentation method that monitors the changes in appearance and topological structure of the manipulated object. These are then used in a visual semantic graph (VSG) based procedure applied to the time sequence of the monitored object to recognize the action consequence. We provide a new dataset, called anipulation Action Consequences (MAC 1.0), which can serve as test bed for other studies on this topic. Several experiments on this dataset demonstrates that our method can robustly track objects and detect their deformations and division during the manipulation. Quantitative tests prove the effectiveness and efficiency of the method.

AB - The problem of action recognition and human activity has been an active research area in Computer Vision and Robotics. While full-body motions can be characterized by movement and change of posture, no characterization, that holds invariance, has yet been proposed for the description of manipulation actions. We propose that a fundamental concept in understanding such actions, are the consequences of actions. There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions. In this paper a technique is developed to recognize these action consequences. At the heart of the technique lies a novel active tracking and segmentation method that monitors the changes in appearance and topological structure of the manipulated object. These are then used in a visual semantic graph (VSG) based procedure applied to the time sequence of the monitored object to recognize the action consequence. We provide a new dataset, called anipulation Action Consequences (MAC 1.0), which can serve as test bed for other studies on this topic. Several experiments on this dataset demonstrates that our method can robustly track objects and detect their deformations and division during the manipulation. Quantitative tests prove the effectiveness and efficiency of the method.

UR - http://www.scopus.com/inward/record.url?scp=84887345046&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887345046&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2013.331

DO - 10.1109/CVPR.2013.331

M3 - Article

AN - SCOPUS:84887345046

SP - 2563

EP - 2570

JO - Scanning Electron Microscopy

JF - Scanning Electron Microscopy

SN - 0586-5581

M1 - 6619175

ER -