MIT prototype lets excavator novices match experts in a simulator

A trainee grips something that looks less like a game controller than a tiny excavator arm. Move it forward, rotate it or close its bucket, and a virtual digger copies the gesture across a six-screen display. In MIT’s test, novice operators matched experts from their very first simulator session—without learning the joystick choreography that normally stands between a person and the machine.

That choreography is the hidden difficulty of excavating. A joystick movement does not resemble the boom, arm, bucket or cab motion it commands, so beginners must build a mental map before they can dig trenches, clear debris or grade soil fluently. Hermano Krebs, a principal research scientist in MIT’s Department of Mechanical Engineering, says the World-Space Interface removes much of that translation by letting people mime the movement they want.

The team compared the device with a conventional joystick simulator using expert and novice volunteers. They trained for one hour a day across seven days in 15 virtual settings, including construction sites, highways, mining areas, riverbanks and urban and rural environments. With joysticks, novices improved but stayed behind experts; with the new interface, that gap disappeared at the start of training.

The practical target is an aging operator workforce, a concern first raised by Japanese machinery manufacturer Sumitomo Heavy Industries when its collaboration with Krebs began in 2018. The same arm-shaped controller could eventually sit beside an operator in a cab, or let someone teleoperate an excavator from a trailer when the worksite is difficult or unsafe. Those uses are still a vision, not a reported deployment: the study tested a virtual excavator.

So what changes for construction companies? If the result holds beyond the simulator, trainees could spend less time learning an abstract control system before becoming useful on basic tasks. The technology is not ready to make that promise yet: the prototype has no haptic feedback, and MIT is still developing force feedback to make the arm resist as if it were scooping heavy material. Caterpillar, Hyundai and Komatsu are also developing excavator simulators, but their systems remain based on traditional joysticks.