Robot hand trained in simulation spins a lightbulb
Researchers taught a robot hand to spin a lightbulb, with no human data.
A robot hand trained in simulation reached 95.2% of its goals on a real hand, and it can spin a lightbulb held in an arbitrary pose.
Learned hand skills remain limited to simple grasps or basic repeated motions, such as rotating an object. A new paper from Ian Chuang, Hiya Shah, Vamsi Chilamkur, Iman Soltani and Claire Tomlin goes after a harder job: moving an object to almost any position and orientation within reach of the fingers, at any wrist orientation, through long sequences of consecutive goals.
The team trained policies for a LEAP hand in simulation without human data. Their recipe includes diverse resets and goals from generated grasp datasets, and an adaptive friction curriculum. With careful system identification, the policies transfer zero-shot to the real hand, using either motion capture or a single wrist-mounted depth camera.
Safety, such as not dropping the object, is rarely measured. The researchers add a learned residual safety filter with adjustable conservativeness, so they can tune the balance between performance and safety.
Real-robot testing ran 13.3 hours over 4,800 unseen goals, across 4 objects. The state-based policy reached 95.2% of goals within 2 cm and 0.4 rad. The filter cut its drops by 75% while the policy still reached 91.5% of goals.
The policies also reach out-of-distribution goals and withstand fast arm motion and perturbations. The team releases the full recipe for others to study.
Would you trust a robot hand to screw in a lightbulb for you?
Video: ian-chuang.github.io/dexsafe