Memory system lifts home-robot task success to 58.9% in simulation

A robot is asked to fetch a book it saw on the coffee table an hour earlier. It has since finished other chores, so it searches every room, or keeps tugging at a drawer it already found locked. That is the everyday failure a team at the OmniAI Group at Zhejiang University wanted to measure. Their benchmark, EmbodiedMemory-Bench (EMem-Bench), tests whether embodied AI agents, meaning software that acts through a body or simulated body, can build memory from long histories of interaction and then act on it.

The researchers first examined 100 failed attempts by two models, Gemini-3-Flash and Qwen3-VL-32B, on the EmbodiedBench test suites. The causes were specific: 35% came from missing fine visual details, 31% from failing to record states revealed by an interaction, 18% from overlooking changes in the environment, and 7% from failing to reuse past experience. The benchmark turns these four gaps into task families, and each of its 2,554 executable episodes gives the agent a time-ordered multimodal history before a new task. The agent must retrieve the right evidence, then navigate, open containers and move objects in the AI2-THOR simulator until the goal is reached. To make the tests harder, single rooms are joined into multi-room homes and distracting but executable interactions are inserted between the key clue and the request.

The team's own answer is Embodied-Memorizer (EMem), an external memory system that stores experience as scene, spatial and event memories. Using GPT-5.4-mini as its backbone, EMem raised average success from 35.3% to 58.9%, which is 21.1 points above TeleMem, the next-best memory system tested. A fine-tuned 8-billion-parameter policy, EMem-8B, reached 46.0%, against a base model score 20.6 points lower. The result also clarifies a common shortcut. Simply feeding more history images did not help: success fell from 36.6% to 26.2%, while input tokens per episode rose from 41,770 to 206,489.

The baselines show how hard the benchmark is. With full context, Gemini-3-Flash led at 64.2%, followed by Gemini-2.5-Pro at 60.8%, Qwen3.6-27B at 58.2% and GPT-5.4 at 56.8%. All but one open-source model scored below 45%. For developers building home robots, the paper suggests that structuring experience can work better than piling on raw history. A benchmark like this also gives teams a shared yardstick, which matters for anyone who wants to compare systems on the same tasks.

The limits are clear, and the authors name them. Every result comes from simulation, while a real robot must cope with sensor noise and execution errors that a virtual kitchen never produces. Until these systems run on physical hardware, the 58.9% figure measures the memory approach, not a robot that can be trusted to fetch the book. The next test is the one the authors themselves point to: testing EMem on physical robots that face sensor noise and execution errors.