Meta AI’s Mac beta reads screens and dictates across apps

A keyboard shortcut brings Meta AI to the Mac without opening a browser. In its beta app, the assistant can read a shared screen, answer questions about what it sees and create content from that context; it also offers system-wide dictation, so spoken text can be inserted into other applications.

The mechanism is deliberately narrower than giving an AI control of the computer. Meta AI can work from screenshots, extracting text and image information, but it does not appear to steer the keyboard or mouse. The app uses Meta’s Muse Spark model, according to TechCrunch, and runs as a dedicated Mac application built with WebKit and SwiftUI, rather than a basic Electron wrapper, according to heise online.

That puts Meta in a desktop race already occupied by Google’s Gemini, OpenAI’s ChatGPT and Anthropic’s Claude. Gemini can also use a shared window, while ChatGPT and Claude go further by allowing their chatbots to take control of a computer. Meta’s release therefore adds a new place to use its assistant, but not the broadest form of computer automation.

The more consequential expansion may be aimed at business owners. Meta says merchants can connect Instagram and Facebook accounts, advertising campaigns and Google Workspace, then ask questions about campaign performance, likes, shares and saves. The assistant can use those account details and information from the web to suggest future posts, draft documents, build decks and spreadsheets, and provide recurring performance updates.

So what changes in practice? A merchant could move from checking several dashboards to asking one assistant which posts performed well and requesting a first draft of the next proposal. That workflow is described by Meta for a beta app: its screen access is for analysis rather than computer control.