Page Agent from Alibaba Manages Web Interfaces via DOM Without Screenshots and Multimodal Models

Alibaba unveiled Page Agent on July 2, 2026 — a browser agent written in pure JavaScript that controls web interfaces through text commands by reading the page structure via DOM. The agent operates entirely on the client side and requires no screenshots, multimodal models, or changes to server code.

How Page Agent Works

The agent is embedded in a web page as a regular JavaScript script. When a user or external system sends a natural language command — for example, "click the form submit button" — the agent analyzes the live DOM tree: it reads tags, attributes, text labels, and nesting structure. Based on this analysis, it selects the required element and performs an action: clicking, text input, or dropdown selection.

Key characteristics:

• Client-side execution — no server requests needed to control the UI
• Control signal — plain natural language text
• Data source — textual representation of the live DOM
• Actions — clicks, input, interaction with form elements
• No multimodal model required — a text-based LLM is sufficient

Why DOM is Better Than Screenshots

Most modern GUI agents for browsers work through visual page representation: take a screenshot, feed the image to a multimodal model, retrieve coordinates, execute a click. This approach works, but has fundamental limitations.

Multimodal requests are more expensive than text requests — both in token costs and response latency. Dense interfaces with small elements reduce accuracy: the model may misidentify coordinates or "lose" a button behind a popup layer. The screenshot approach also depends on rendering — the agent behaves differently at different window sizes and in headless mode.

An additional problem is dynamic content. If an element appears hundreds of milliseconds after page load, a screenshot-based agent must guess the stabilization moment. A DOM agent retrieves the current state of the tree at the moment of the command and sees fresh content automatically.

The DOM approach eliminates all these limitations. The HTML structure of the page already contains semantics — identifiers, roles, text labels, element hierarchy. A text agent can more accurately find "the confirmation button in the modal window" by reading the DOM than by guessing an object from pixels. The agent works equally well in regular Chrome, mobile WebView, and headless browsers.

Where This Is Applicable

The DOM agent shows the best results where page structure is predictable: enterprise web applications, CRM systems, administrative panels, forms. It is fundamentally important that Page Agent requires no access to server code or the host site's API. The service owner does not need to rewrite anything — simply connecting the client-side agent script is sufficient.

This opens a "zero-integration" scenario: a developer or end user adds the agent to any already-running website and immediately begins controlling it with text commands. For enterprise automation, this significantly reduces implementation costs compared to traditional RPA solutions that require deep customization for each specific application.

What This Means

Alibaba's Page Agent demonstrates that browser automation does not require expensive multimodal models. The DOM as a text source provides the agent with sufficient information for precise interface control — at lower costs and without dependence on visual rendering. If this approach proves reliable across diverse real-world websites, it could set a new standard for lightweight web agents — embeddable without backend rewriting and compatible with any website out of the box.