Alibaba's Page Agent Controls Web Interfaces Through the DOM, With No Screenshots or Multimodal Models

On July 2, 2026, Alibaba introduced Page Agent, a browser agent written in plain JavaScript that controls web interfaces from text commands by reading the page structure through the DOM. The agent runs entirely on the client side and requires no screenshots, no multimodal models and no changes to server-side code.

How Page Agent works

The agent is embedded in a web page as an ordinary JS script. When a user or an external system passes it a natural-language command, such as "click the form submit button," the agent analyzes the live DOM tree: it reads tags, attributes, text labels and nesting structure. Based on that analysis, it picks the right element and performs the action: a click, text input or a selection from a dropdown list.

Key characteristics:

• Client-side execution: no server requests are needed to control the UI
• Control signal: plain natural-language text
• Data source: a text representation of the live DOM
• Actions: clicks, typing, interaction with form elements
• No multimodal model required: a text-only LLM is enough

Why is the DOM better than a screenshot?

Most current GUI agents for the browser work from a visual representation of the page: take a screenshot, feed the image to a multimodal model, get coordinates back, perform the click. This approach works, but it has systemic limitations.

Multimodal requests cost more than text ones, both in token cost and in response latency. Dense interfaces with small elements reduce accuracy: the model can get the coordinates wrong or "lose" a button behind a pop-up layer. The screenshot approach also depends on rendering, so the agent behaves differently at different window sizes and in headless mode.

Dynamic content is another problem. If an element appears a few hundred milliseconds after load, a screenshot-based agent has to guess when the page has settled. A DOM agent takes the current state of the tree at the moment of the command and sees fresh content automatically.

The DOM approach removes all of these limitations. A page's HTML structure already carries semantics: identifiers, roles, text labels and element hierarchy. For a text agent, finding the "confirm button in the modal window" is more accurate when it reads the DOM than when it guesses the object from pixels. The agent behaves the same in regular Chrome, a mobile WebView and a headless browser.

Where it applies

A DOM agent performs best where page structure is predictable: enterprise web applications, CRM systems, admin panels and forms. Crucially, Page Agent does not need access to the host site's server code or API. The service owner does not have to rewrite anything; it is enough to add the agent script on the client side.

This opens up a "zero integration" scenario: a developer or end user adds the agent to any already running site and immediately starts controlling it with text commands. For enterprise automation, this substantially lowers the cost of deployment compared with traditional RPA solutions, which require deep configuration for each specific application.

What this means

Alibaba's Page Agent shows that browser automation does not require expensive multimodal models. The DOM as a text source gives the agent enough information to control an interface accurately, at lower cost and without depending on visual rendering. If the approach proves reliable across diverse real-world sites, it could set a new standard for lightweight web agents that can be embedded without rewriting the backend and work with any site out of the box.