TY - GEN
T1 - Manipulation action tree bank
T2 - 2014 14th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2014
AU - Yang, Yezhou
AU - Guha, Anupam
AU - Fermüller, Cornelia
AU - Aloimonos, Yiannis
PY - 2015/2/12
Y1 - 2015/2/12
N2 - Our premise is that actions of manipulation are represented at multiple levels of abstraction. At the high level a grammatical structure represents symbolic information (objects, actions, tools, body parts) and their interaction in a temporal sequence, and at lower levels the symbolic quantities are grounded in perception. In this paper we create symbolic high-level representations in the form of manipulation action tree banks, which are parsed from annotated action corpora. A context free grammar provides the grammatical description for the creation of the semantic trees. Experiments conducted on the tree banks show that they allow to 1) generate so-called visual semantic graphs (VSGs), 2) compare the semantic distance between steps of activities and 3) discover the underlying semantic space of an activity. We believe that tree banks are an effective and practical way to organize semantic structures of manipulation actions for humanoids applications. They could be used as basis for 1) automatic manipulation action understanding and execution and 2) reasoning and prediction during both observation and execution. The knowledge resource follows the widely used Penn Tree Bank format.
AB - Our premise is that actions of manipulation are represented at multiple levels of abstraction. At the high level a grammatical structure represents symbolic information (objects, actions, tools, body parts) and their interaction in a temporal sequence, and at lower levels the symbolic quantities are grounded in perception. In this paper we create symbolic high-level representations in the form of manipulation action tree banks, which are parsed from annotated action corpora. A context free grammar provides the grammatical description for the creation of the semantic trees. Experiments conducted on the tree banks show that they allow to 1) generate so-called visual semantic graphs (VSGs), 2) compare the semantic distance between steps of activities and 3) discover the underlying semantic space of an activity. We believe that tree banks are an effective and practical way to organize semantic structures of manipulation actions for humanoids applications. They could be used as basis for 1) automatic manipulation action understanding and execution and 2) reasoning and prediction during both observation and execution. The knowledge resource follows the widely used Penn Tree Bank format.
UR - http://www.scopus.com/inward/record.url?scp=84945184673&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84945184673&partnerID=8YFLogxK
U2 - 10.1109/HUMANOIDS.2014.7041483
DO - 10.1109/HUMANOIDS.2014.7041483
M3 - Conference contribution
AN - SCOPUS:84945184673
T3 - IEEE-RAS International Conference on Humanoid Robots
SP - 987
EP - 992
BT - 2014 IEEE-RAS International Conference on Humanoid Robots, Humanoids 2014
PB - IEEE Computer Society
Y2 - 18 November 2014 through 20 November 2014
ER -