Remote robot control with human-in-the-loop over long distances using digital twins

Ievgenii A. Tsokalo, David Kuss, Ievgen Kharabet, Frank H.P. Fitzek, Martin Reisslein

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The sharing of skills over the Internet enables professionals to democratize their expertise and skills without exhausting their availability, e.g., through excessive traveling. To enable this Internet of Skills, we present a novel Digital Twin (DT) platform for the remote control of machines with human-in-the-loop. The DT of a remotely controlled machine acts effectively as an inter-layer between the operator and the controlled machine, e.g., robot arm. The DT can be optimized for a particular application to interact with the operator with an intuitive low-latency interface and, on other side, to control and monitor the quality of the remote task. Essentially, the human operator controls the DT, while the DT controls the remote robot. This paper introduces the DT framework for the remote control. The human-machine-human control loop is split into Virtual Reality (VR), remote control, and robot control loops. The proposed framework achieves low latency visual feedback and very short system reaction times for unexpected changes with arbitrary distances between operator and robot. Within the DT framework, this paper proposes a robot control algorithm for controlling time-critical robot applications over networks with considerable delays and jitter. The proposed framework has been implemented in a demonstrator with a robot arm and its DT in VR.

Original languageEnglish (US)
Title of host publication2019 IEEE Global Communications Conference, GLOBECOM 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728109626
DOIs
StatePublished - Dec 2019
Externally publishedYes
Event2019 IEEE Global Communications Conference, GLOBECOM 2019 - Waikoloa, United States
Duration: Dec 9 2019Dec 13 2019

Publication series

Name2019 IEEE Global Communications Conference, GLOBECOM 2019 - Proceedings

Conference

Conference2019 IEEE Global Communications Conference, GLOBECOM 2019
CountryUnited States
CityWaikoloa
Period12/9/1912/13/19

Keywords

  • Closed-loop control
  • Edge cloud
  • Low latency
  • Remote control
  • Robot arm
  • Tactile Internet

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems
  • Signal Processing
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Media Technology
  • Health Informatics

Fingerprint Dive into the research topics of 'Remote robot control with human-in-the-loop over long distances using digital twins'. Together they form a unique fingerprint.

  • Cite this

    Tsokalo, I. A., Kuss, D., Kharabet, I., Fitzek, F. H. P., & Reisslein, M. (2019). Remote robot control with human-in-the-loop over long distances using digital twins. In 2019 IEEE Global Communications Conference, GLOBECOM 2019 - Proceedings [9013428] (2019 IEEE Global Communications Conference, GLOBECOM 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/GLOBECOM38437.2019.9013428