Mind the gap: towards building a more seamless sim-to-real platform for robotic grasping and manipulation

Zhengshen Zhang; Haozhe Wang; Lei Zhou; Zhiyang Liu; Chenchen Liu; Francis EH Tay; Wen Feng Lu; Marcelo H. Ang Jr.

doi:10.1117/12.3023745

14 March 2024 Mind the gap: towards building a more seamless sim-to-real platform for robotic grasping and manipulation

Zhengshen Zhang, Haozhe Wang, Lei Zhou, Zhiyang Liu, Chenchen Liu, Francis EH Tay, Wen Feng Lu, Marcelo H. Ang Jr.

Proceedings Volume 13074, Fifth International Conference on Image, Video Processing, and Artificial Intelligence (IVPAI 2023); 130740L (2024) https://doi.org/10.1117/12.3023745
Event: Fifth International Conference on Image, Video Processing and Artificial Intelligence, 2023, Shenzhen, China

Abstract

This paper presents a vision of creating a more seamless sim-to-real robotic grasping and manipulation platform, bridging the gap between simulated environments and real-world applications. In recent years, the emergence of digital twin technology has revolutionized how we develop and test such applications. A digital twin can simulate the behavior of a physical system in a virtual replica of the real-world environment in real-time, enabling engineers and researchers to conduct detailed analysis and evaluation before deploying robotic systems in the real world. In this paper, we study the potential of creating a digital twin that allows researchers and engineers to seamlessly deploy a grasping system trained on synthetic data and tested in a simulation environment without writing any additional code or doing additional calibration. Using our proposed platform, we present a case study done using a robotic grasping algorithm and analyze the advantages and limitations of our current platform. In the end, we suggest some possible improvements and future directions to enhance the platform’s effectiveness and applicability.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Zhengshen Zhang, Haozhe Wang, Lei Zhou, Zhiyang Liu, Chenchen Liu, Francis EH Tay, Wen Feng Lu, and Marcelo H. Ang Jr. "Mind the gap: towards building a more seamless sim-to-real platform for robotic grasping and manipulation", Proc. SPIE 13074, Fifth International Conference on Image, Video Processing, and Artificial Intelligence (IVPAI 2023), 130740L (14 March 2024); https://doi.org/10.1117/12.3023745

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
5 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Robotics

Computer simulations

Robots

Education and training

Robotic systems

Virtual reality

Control systems

Show All Keywords

Keywords/Phrases

Search In:

Publication Years