MIT system explains robotaxi decisions in private-track tests

As a Motional robotaxi approached a cyclist on a private track, it consistently stopped. A new system developed by MIT and Motional supplied a real-time explanation alongside the vehicle’s planned trajectory, giving the human safety driver more than the bare fact of a stop.

That matters because the planner—the software that decides what the car should do next—works as the vehicle’s brain. It processes data from cameras and lidar sensors, builds a high-level picture of the surroundings, selects an action and outputs a trajectory. Deep-learning planners can be difficult to inspect, leaving people uncertain about unexpected behavior such as phantom braking or a stop that could block an oncoming emergency vehicle.

The system, called the Concept-Wrapper Network, or CW-Net, is a concept classifier: an AI module trained to identify high-level ideas in the vehicle’s input. It is placed inside the existing planner and makes the final part of that planner use concepts such as “approaching stopped vehicle” and “close to cyclist.” That link is intended to keep the explanation faithful to the actual decision rather than producing a plausible story after the fact. The researchers designed CW-Net to mimic the planner’s driving decisions, so it would not negatively affect performance.

The researchers trained CW-Net on 130 million examples of self-driving scenes, each containing multiple labeled concepts. In the private-track test, the paper reports that safety drivers became better at predicting the robotaxi’s behavior in surprising situations. A larger simulation with nonexpert users produced similar results. Julie Shah, an MIT professor and co-senior author, said explanations can help people build a more accurate mental model of the system.

Concretely, the immediate benefit is not a driverless car suddenly becoming error-free. It is better information for the people who must supervise the vehicle and for engineers who may use the live explanations to troubleshoot its artificial-intelligence systems. The work remains at the testing stage: the reported evidence comes from a private-track robotaxi and simulation, while stronger safety and transparency benefits are described as possibilities for the longer term.