SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation

In order to successfully manipulate physical objects, robots need to deduce their physical properties. Vision-based methods require structured environments and have limited applications in real-world scenarios. A recent paper suggests employing tactile sensing to infer the physical parameters of an unknown object.

Image Credit: NASA

Firstly, two in-hand exploration actions are performed: tilting and shaking. Information from both actions is fused to learn a joint physical feature embedding. A swing-up angle predictor finds optimal control parameters using the learned information to swing-up the object to the desired pose.

The results show that the task is accomplished with an overall 17.2-degree error. The suggested approach outperforms other methods that do not use tactile information. It is demonstrated that the learned embedding also could be used to regress properties like mass, the moment of inertia, or friction.

Several robot manipulation tasks are extremely sensitive to variations of the physical properties of the manipulated objects. One such task is manipulating objects by using gravity or arm accelerations, increasing the importance of mass, center of mass, and friction information. We present SwingBot, a robot that is able to learn the physical features of a held object through tactile exploration. Two exploration actions (tilting and shaking) provide the tactile information used to create a physical feature embedding space. With this embedding, SwingBot is able to predict the swing angle achieved by a robot performing dynamic swing-up manipulations on a previously unseen object. Using these predictions, it is able to search for the optimal control parameters for a desired swing-up angle. We show that with the learned physical features our end-to-end self-supervised learning pipeline is able to substantially improve the accuracy of swinging up unseen objects. We also show that objects with similar dynamics are closer to each other on the embedding space and that the embedding can be disentangled into values of specific physical properties.

Research paper: Wang, C., Wang, S., Romero, B., Veiga, F., and Adelson, E., “SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation”, 2021. Link: https://arxiv.org/abs/2101.11812


Notify of
Inline Feedbacks
View all comments
Would love your thoughts, please comment.x