Robots learn to climb from phone videos in the lab

A box, a human body balancing, multiple points of contact with the ground: for a robot, these add up to a chain of small uncertainties. A team from the Robotics & AI Institute and Boston Dynamics has now demonstrated a method that enables robots to learn climbing, cartwheels and backflips from motion data, videos or animations—directly on the hardware.

Many specific movement sequences previously had to be trained individually. The new approach bundles these examples: instead of specifying each movement separately, the machines learn skills from existing motion material. This can also include videos recorded with a phone. The method was presented in the specialist journal Science Robotics.

The difficulty lies not only in the movement itself. Human actions require dexterity, agility and coordination; at the same time, contact with the ground changes. Varying ground conditions and inaccuracies in the environment make control more difficult. That is precisely why human-like movements are considered particularly challenging in robotics.

What next? If the approach continues to be validated, training humanoid robots could become more versatile: new movement skills could be derived from data, videos or animations instead of training every individual sequence from scratch. This would be particularly relevant for robots that are expected to take on everyday tasks in the future. For now, it remains a research project.