SenseTime Open-Sources Robot-AI Test Kit and 27B Model

In one of SenseTime's tests, stacking bowls took a model about 131 minutes and roughly $19 in estimated inference costs, the price of running the model on the task. Putting a pen into a pen holder took 263 minutes and about $55. Those were the bills for pushing GPT-6-Astra, a large AI model, to succeed on robot tasks once it was given training demonstrations and more steps. SenseTime has now put the tools for measuring that gap online, along with a model of its own.

SenseTime has open-sourced SenseNova-RoboRSI, a framework for evaluating embodied AI, meaning AI systems that control a physical body, here robots. It ships with a companion model, SenseNova-RoboRSI-27B-Preview, where 27B refers to a size of 27 billion parameters, the adjustable values inside a model. The framework packages each benchmark, a standardized test, and its dependencies into a separate Docker image, a sealed software container. It then exposes simulation and scoring through a unified HTTP API, the standard way programs exchange data over the web. SenseTime says researchers can run end-to-end evaluations from a configuration file, without installing each simulator by hand.

The extended-budget results are where the study gets concrete. With training demonstrations and step limits raised to as much as 10 times the official settings, GPT-6-Astra saturated the LIBERO and SIMPLE benchmarks, meaning it hit their ceiling, and reached an 81.9% success rate on RoboDojo. In one case, nine RoboDojo tasks that had failed in all five attempts succeeded in all five once the model was allowed to keep interacting. The same study found its limits: the model remained weakest on long-horizon tasks, struggled to transfer skills from one task to another, and had trouble with deformable objects and fine control on real robots.

SenseTime then paired a slow reasoning model with a fast action model. The best combination reached 54% success across 42 RoboDojo tasks, using a closed loop that checks each result before the next step. SenseTime says cost remains an issue.

The 27B model aims to bake tested experience into its weights, the internal values that define a model, so that it needs less trial and error. SenseTime reports it scored 26.13 on RoboDojo with a 20.83% task success rate, topped 90 on LIBERO-Pro, and raised its RoboCasa365 success rate from 26.0% to 46.5% while cutting token use by 73%. Tokens are the small chunks of text a model processes, so fewer tokens means less computation per task. The company says it completes embodied tasks at one-fiftieth of GPT-6-Astra's cost.

For robotics teams, the change is practical and immediate: the test bench is downloadable now, so two labs can run the same benchmarks from the same configuration file instead of rebuilding each simulator. SenseTime also argues that future benchmarks should be built around real human needs, that robots should learn by interacting, and that thinking time should count as world time, since a pan keeps heating while a model deliberates. None of these numbers has been independently verified, and the cost per task, which the company itself flags, is still the open problem.