Sim2Real<sup>2</sup> Actively Building Explicit Physics Model for Precise Articulated Object Manipulation

Sim2Real²: Actively Building Explicit Physics Model for Precise Articulated Object Manipulation

Liqian Ma^*, Jiaojiao Meng^*, Shuntao Liu, Weihang Chen, Jing Xu, and Rui Chen

Tsinghua University
ICRA 2023
^*Indicates Equal Contribution

Abstract

Accurately manipulating articulated objects is a challenging yet important task for real robot applications. In this paper, we present a novel framework called Sim2Real² to enable the robot to manipulate an unseen articulated object to the desired state precisely in the real world with no human demonstrations. We leverage recent advances in physics simulation and learning-based perception to build the interactive explicit physics model ofthe object and use it to plan a long-horizon manipulation trajectory to accomplish the task. However, the interactive model cannot be correctly estimated from a static observation. Therefore, we learn to predict the object affordance from a single-frame point cloud, control the robot to actively interact with the object with a one-step action, and capture another point cloud.Further, the physics model is constructed from the two point clouds. Experimental results show that our framework achieves about 70% manipulations with 30% relative error for common

Results

Results of real-world articulated object manipulation experiments. Each row shows the result of 3 categories respectively. The left column shows the error of the manipulation. The right column shows the relative error of the manipulation. The sign of the target movement denotes the direction of the movement (e.g. opening or closing, clockwise (CW) or counterclockwise (CCW).

Results of ablation study on reward function. The 5 tasks are opening/closing drawer, opening/closing laptop and turning faucet. The task is considered to be failed if it is not done when time step reaches 50.

Ablation study on Interactive Perception. The model constructed with Interactive Perception (IP) has more accurate movable part segmentation and joint axis estimation than without IP.

Open drawer with tools. In real scenarios, the object may be beyond the robot's reach, or the gripper cannot fit into the object's size. Our method can be extended to tool-using cases. As shown in these two sequences, the robot uses a T-shaped tool or a semi-ring to open the small drawer.

BibTeX

@INPROCEEDINGS{10160370, author={Ma, Liqian and Meng, Jiaojiao and Liu, Shuntao and Chen, Weihang and Xu, Jing and Chen, Rui}, booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)}, title={Sim2Real2: Actively Building Explicit Physics Model for Precise Articulated Object Manipulation}, year={2023}, volume={}, number={}, pages={11698-11704}, doi={10.1109/ICRA48891.2023.10160370}}

Sim2Real2: Actively Building Explicit Physics Model for Precise Articulated Object Manipulation