Qwen-UI-Agent scores 97.5 percent on AndroidDaily benchmark

A phone screen fills with apps, menus and half-finished tasks. Qwen-UI-Agent is built to move through them: Alibaba’s Qwen released the model on August 20 to operate phones, desktops, websites and deep search by clicking, typing and swiping. In one demonstration, it looked up information in Amap, summarized Xiaohongshu posts and booked through DingTalk across five or six apps.

The approach focuses on real devices. Many graphical-user-interface agents — systems that operate software through what appears on screen — are trained in simulators, while Alibaba says it trained and evaluated this model on real devices. Alibaba instead built a mobile environment with more than 100 real phones and 150-plus apps, then created the 400-task MobileWorld-Real benchmark. The model reached 97.5 percent on AndroidDaily and 82.1 percent on MobileWorld; on desktops, Alibaba reports 79.5 percent on OSWorld-Verified and 73.6 percent on WebArena.

Qwen-UI-Agent is not limited to mouse clicks. It can use CLI, the command-line interface used to run software through text commands, for nearly half of its desktop actions. In a 170-step task, it cross-checked financial data, ran analysis scripts, created Excel, PowerPoint and Word files, and used visual checks to repair layouts. Online reinforcement learning — training from the results of ongoing attempts — supports trajectories beyond 100 steps, with about 10,000 concurrent environments.

The more useful detail may be where the agent stops. For a request to send a mother 500 yuan through Alipay, it fills in the amount and note but pauses at payment. It also asks for missing flight dates or cabin class, and waits before sending red envelopes, deleting files or granting privacy authorization. For illegal or high-risk requests, Alibaba says it performs no operation and terminates instead.

So what changes, concretely? A user could hand over a tangled task that crosses apps rather than waiting for each application to expose an API: a flight-cancellation notice could trigger a proposed rebooking plan, or a receipt could move from a phone’s photo library to a desktop folder before an Excel bill summary is generated. Qwen-UI-Agent is also designed to hand work between phone and desktop. For now, these are demonstrations and benchmark results, not proof of a consumer assistant in service; Alibaba has made the technical report, project page and code public for others to inspect.