Tagline
The web was built for people. tldr is a renderer built for agents: half the tokens, same accuracy.
Inspiration
Computer use is becoming one of the main ways agents get work done, and the web is where most of that work is. But everything an agent can read today was designed for someone else. Web pages are designed for humans, HTML for browsers, and accessibility trees for screen readers. An agent using any of them is like a person searching for a movie title with a Roku remote: no matter how smart you are, the interface sets the ceiling. We wanted to build the interface that the web never had, a renderer whose output is meant to be read by an agent.
What it does
tldr is an alternative renderer for the web. It takes a page and produces a representation built around what an agent actually does on a page, which is act on things:
- Anything a user could act on gets a line and an ID. Links, buttons, inputs, selects, checkboxes, menu items, tabs, and anything the page marks clickable stay visible, named, with their state.
- Passive content becomes a summary. Any subtree with nothing to act on folds into a single line, so a product description or footer costs a dozen tokens instead of hundreds.
- Dropdowns become one line. A
<select>with forty options is shown as the control and its current value, not forty lines. - Labels are attached to their inputs, so a field reads as "Email" rather than an unnamed textbox next to some text.
- Every ID maps back to a DOM node, so whatever the agent picks can be executed directly.
It is deterministic and has no model inside. The only cost it adds is a pass over the accessibility tree, which is small next to the model call; it behaves the same on every site and can replace the accessibility-tree serializer under any agent framework.
How we built it
We started by running agents on the standard representation, Chromium's full accessibility tree as an indented outline, the way Playwright exposes it. Two things stood out in the traces: most of the content was noise (generic wrappers, repeated static text, option lists), and when the agent failed, the right target was usually there on the page. Those observations became the seed of an automated research loop. Following Karpathy's Auto Research pattern, the agent proposed candidate translators, implemented them as passes over a shared intermediate representation, screened them cheaply, and promoted survivors to full runs. Every trace of every run went in full to Universal Constructor, where the napa agent wrote up what separated failures from successes and batched facet queries measured each hypothesis across all 300 pages in seconds. Every lead was verified against the raw HTML before it was built. We evaluated on Mind2Web (single-turn replays of humans on 137 real websites; 300/300/900 train/val/test) and WebArena (multi-turn tasks on self-hosted sites; 40/160).
Challenges we ran into
We spent a lot of the project trying to give the agent a hierarchy to navigate instead of a flat page: a node-at-a-time tree, a segment tree with summaries at every level and a FIND command, and a two-level index with a top-level view that expands on request. None of them worked. Whenever the agent had to choose a branch from a summary rather than see the element, its ability to find a known element fell from 80–85% on the flat page to 29–59%, and making the summaries detailed enough to fix that brought the size back to the flat page. We dropped the idea and kept the flat index, complete over actions and lossy only over content.
The second challenge was accepting where the remaining failures live. With a good representation the agent finds a described element 85–89% of the time; what it gets wrong is which step to take next, and about half of those are defensible alternatives. The interface fixes cost, not planning, and we stopped chasing accuracy changes that measured as noise.
Accomplishments that we're proud of
- Agents get cheaper. Half the tokens at the same accuracy. On Mind2Web held-out, same reader and prompt, tldr matches the accessibility-tree baseline on step success at 48% fewer input tokens and 47% lower cost per action. On an average multi-turn WebArena task, cost fell 44%, from $0.034 to $0.019.
- Pages get smaller. An average Mind2Web page is about 6,800 tokens (~600 lines) instead of about 13,800 tokens (~1,500 lines), and the agent can still act on everything it could before.
- A rule for agent-native rendering. Complete over actions, lossy over content, IDs as pointers. We measured why the alternatives fail, so the rule is a result, not a preference.
- The web's defects show up when you build for agents. Mind2Web stores ARIA labels in a form Chromium ignores, so icon buttons were nameless in every published baseline; restoring them raised element grounding by six points.
What we learned
- Interfaces built for humans or browsers are not neutral for agents; the representation sets a hard ceiling on cost and a soft one on accuracy.
- An agent reader needs a complete index of what it can act on. Hierarchies it must navigate cost more than they save.
- Measure finding and planning separately; only one of them belongs to the renderer.
What's next for tldr
- Make tldr the agent's only view of the web. Today tldr is a translator you call on HTML. We want to release a browser harness in which the agent never sees the accessibility tree at all: it observes the tldr index, acts by ID, and the harness resolves the ID to the DOM node and executes. Our demo already runs two agents side by side on the same task, one on the accessibility tree and one on tldr; the tldr side should become a standalone Playwright package that frameworks like browser-use can adopt as their default page view.
- Apply the same renderer to the other interfaces that were built for people. Desktop and mobile apps expose the same kind of tree the browser does (UIAutomation, macOS AX, Android's accessibility service), with the same wrappers and static text around a few actionable controls, so the passes transfer with a new frontend. Terminal output is the other case we want to try: build logs and test output are rendered for a scrolling screen, and an agent reads thousands of lines to find the few that name a failing test or a file path. The rule is the same in both, complete over actions and lossy over content, applied to a new source.
- Render change, not pages. In a multi-turn task the agent re-reads the whole page every step although most of it did not change. Showing the full index once and then only what changed is where the remaining cost in long tasks sits.
Built With
- autoresearch
- devin
- mind2web
- openai-gpt-5.4-mini
- playwright
- playwright/chromium
- universal-constructor