Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision

Yiwei Wang, Xin Ye, Yezhou Yang, Wenlong Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

We present a framework from vision based hand movement prediction in a real-world human-robot collaborative scenario for safety guarantee. We first propose a perception submodule that takes in visual data solely and predicts human collaborator's hand movement. Then a robot trajectory adaptive planning submodule is developed that takes the noisy movement prediction signal into consideration for optimization. To validate the proposed systems, we first collect a new human manipulation dataset that can supplement the previous publicly available dataset with motion capture data to serve as the ground truth of hand location. We then integrate the algorithm with a six degree-of-freedom robot manipulator that can collaborate with human workers on a set of trained manipulation actions, and it is shown that such a robot system outperforms the one without movement prediction in terms of collision avoidance. We verify the effectiveness of the proposed motion prediction and robot trajectory planning approaches in both simulated and physical experiments. To the best of the authors' knowledge, it is the first time that a deep model based movement prediction system is utilized and is proven effective in human-robot collaboration scenario for enhanced safety.

Original languageEnglish (US)
Title of host publication2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017
PublisherIEEE Computer Society
Pages305-310
Number of pages6
ISBN (Electronic)9781538646786
DOIs
StatePublished - Dec 22 2017
Event17th IEEE-RAS International Conference on Humanoid Robotics, Humanoids 2017 - Birmingham, United Kingdom
Duration: Nov 15 2017Nov 17 2017

Other

Other17th IEEE-RAS International Conference on Humanoid Robotics, Humanoids 2017
CountryUnited Kingdom
CityBirmingham
Period11/15/1711/17/17

Fingerprint

Human robot interaction
End effectors
Trajectories
Robots
Planning
Collision avoidance
Manipulators
Data acquisition
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Human-Computer Interaction
  • Electrical and Electronic Engineering

Cite this

Wang, Y., Ye, X., Yang, Y., & Zhang, W. (2017). Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision. In 2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017 (pp. 305-310). IEEE Computer Society. https://doi.org/10.1109/HUMANOIDS.2017.8246890

Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision. / Wang, Yiwei; Ye, Xin; Yang, Yezhou; Zhang, Wenlong.

2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017. IEEE Computer Society, 2017. p. 305-310.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, Y, Ye, X, Yang, Y & Zhang, W 2017, Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision. in 2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017. IEEE Computer Society, pp. 305-310, 17th IEEE-RAS International Conference on Humanoid Robotics, Humanoids 2017, Birmingham, United Kingdom, 11/15/17. https://doi.org/10.1109/HUMANOIDS.2017.8246890
Wang Y, Ye X, Yang Y, Zhang W. Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision. In 2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017. IEEE Computer Society. 2017. p. 305-310 https://doi.org/10.1109/HUMANOIDS.2017.8246890
Wang, Yiwei ; Ye, Xin ; Yang, Yezhou ; Zhang, Wenlong. / Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision. 2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017. IEEE Computer Society, 2017. pp. 305-310
@inproceedings{e82b6e8625684ba58d612aa6fca46711,
title = "Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision",
abstract = "We present a framework from vision based hand movement prediction in a real-world human-robot collaborative scenario for safety guarantee. We first propose a perception submodule that takes in visual data solely and predicts human collaborator's hand movement. Then a robot trajectory adaptive planning submodule is developed that takes the noisy movement prediction signal into consideration for optimization. To validate the proposed systems, we first collect a new human manipulation dataset that can supplement the previous publicly available dataset with motion capture data to serve as the ground truth of hand location. We then integrate the algorithm with a six degree-of-freedom robot manipulator that can collaborate with human workers on a set of trained manipulation actions, and it is shown that such a robot system outperforms the one without movement prediction in terms of collision avoidance. We verify the effectiveness of the proposed motion prediction and robot trajectory planning approaches in both simulated and physical experiments. To the best of the authors' knowledge, it is the first time that a deep model based movement prediction system is utilized and is proven effective in human-robot collaboration scenario for enhanced safety.",
author = "Yiwei Wang and Xin Ye and Yezhou Yang and Wenlong Zhang",
year = "2017",
month = "12",
day = "22",
doi = "10.1109/HUMANOIDS.2017.8246890",
language = "English (US)",
pages = "305--310",
booktitle = "2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision

AU - Wang, Yiwei

AU - Ye, Xin

AU - Yang, Yezhou

AU - Zhang, Wenlong

PY - 2017/12/22

Y1 - 2017/12/22

N2 - We present a framework from vision based hand movement prediction in a real-world human-robot collaborative scenario for safety guarantee. We first propose a perception submodule that takes in visual data solely and predicts human collaborator's hand movement. Then a robot trajectory adaptive planning submodule is developed that takes the noisy movement prediction signal into consideration for optimization. To validate the proposed systems, we first collect a new human manipulation dataset that can supplement the previous publicly available dataset with motion capture data to serve as the ground truth of hand location. We then integrate the algorithm with a six degree-of-freedom robot manipulator that can collaborate with human workers on a set of trained manipulation actions, and it is shown that such a robot system outperforms the one without movement prediction in terms of collision avoidance. We verify the effectiveness of the proposed motion prediction and robot trajectory planning approaches in both simulated and physical experiments. To the best of the authors' knowledge, it is the first time that a deep model based movement prediction system is utilized and is proven effective in human-robot collaboration scenario for enhanced safety.

AB - We present a framework from vision based hand movement prediction in a real-world human-robot collaborative scenario for safety guarantee. We first propose a perception submodule that takes in visual data solely and predicts human collaborator's hand movement. Then a robot trajectory adaptive planning submodule is developed that takes the noisy movement prediction signal into consideration for optimization. To validate the proposed systems, we first collect a new human manipulation dataset that can supplement the previous publicly available dataset with motion capture data to serve as the ground truth of hand location. We then integrate the algorithm with a six degree-of-freedom robot manipulator that can collaborate with human workers on a set of trained manipulation actions, and it is shown that such a robot system outperforms the one without movement prediction in terms of collision avoidance. We verify the effectiveness of the proposed motion prediction and robot trajectory planning approaches in both simulated and physical experiments. To the best of the authors' knowledge, it is the first time that a deep model based movement prediction system is utilized and is proven effective in human-robot collaboration scenario for enhanced safety.

UR - http://www.scopus.com/inward/record.url?scp=85044448781&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044448781&partnerID=8YFLogxK

U2 - 10.1109/HUMANOIDS.2017.8246890

DO - 10.1109/HUMANOIDS.2017.8246890

M3 - Conference contribution

AN - SCOPUS:85044448781

SP - 305

EP - 310

BT - 2017 IEEE-RAS 17th International Conference on Humanoid Robotics, Humanoids 2017

PB - IEEE Computer Society

ER -