Flash

If an agent takes 15 minutes to book what you'd book in 5, it's pointless. Flash does the flight search in 5.9 seconds on a desktop you can watch, and watches fares for you after with scripts.

Comment

Inspiration

An agent that takes 15 minutes to book dinner or a flight you could book yourself in 5 is pointless. If you have to wait on it, you didn't save anything, you just moved the work to watching a screen.

The slow part was never the clicking. It was the thinking about the clicking: one model call per action, every action. So I wanted to see how far that could be cut down.

What it does

Flash is an open computer-use agent. You text it, talk to it, or type, and it drives a real Linux desktop you can watch, and finishes the job. Text it on iMessage (through Photon) or Telegram, and it replies with the answer and a screenshot of the desktop.

On "cheapest nonstop RDU to NYC next Friday morning" on live Google Flights, Flash's median is 5.9 seconds, about five cents a run. Grok Bot took 5:36, 7:21 and 5:14 on the same task, a median of 336 seconds. That's about 57x faster at the medians (n = 3 each; timed by hand, products not models, three runs each, answers of Grok Bot not checked for correctness).

It can also watch a fare. Say "tell me if it drops under $X" and Flash compiles what it just did into a Playwright script, then replays it on a timer with zero model calls until the price moves.

Flash stops one click before the pay button, in every test and demo. Grok Bot was told to stop before paying too.

Try it: v0.1.0 release, or install in one line:

curl -fsSL https://flash.teddytennant.com/install.sh | sh

How we built it

Routing with decisions models. Decision models like TypeSafe's Jev got me thinking this way, and OpenAI's Decisions API is the version I went with because I think it's better: it did better on the benchmarks I looked at (Flash supports both behind one interface, but I only ran OpenAI's), it takes text and images, and it's cheap enough to run on every message and every answer. One call to gpt-6-luna scores a handful of yes/no questions: does this need doing now, is it a watch, is something missing, do they want to buy. It routed 58 of 58 labeled prompts, and 42 of 42 rephrasings of them, correctly. I used predicates, not choice, because choice is overconfident and cares about option order.

Done means done. The same API checks the result: right route, date, nonstop, time window, cheapest shown, on the results page. Each has to score 0.9 or higher, and a regex has to find a price and a departure time. A wrong answer can't be called finished.

The agent. gpt-6.1-sol on Ultrafast over a persistent Responses WebSocket. It sees a numbered element table built from the accessibility tree and returns batched actions, so a whole form is one turn. The same ~290-token reply took 2.44 s on Ultrafast against 7.68 s on default (median of 3).

The desktop. A Docker image with Ubuntu 24.04, Xfce on Xvfb, AT-SPI and a real non-headless Chromium, attached over CDP from a warm pool in about 240 ms. You watch it live through noVNC.

Scripts. A run compiles to a Playwright script. When a locator breaks, one snapshot and one model call re-pick the element and patch the script.

Clients. A React web app and an Electron desktop app, both talking to one Fastify daemon. Local, or a server you run, over SSH. Android app released (APK); iPhone works through iMessage today, a native iOS app isn't built yet.

Challenges we ran into

Getting an agent to be right and fast at once. Six of my first nine runs failed the done gate. Google's "Cheapest from $185" tab shows a stale price while fares are still loading, and the agent believed it. The gate caught every one. I fixed it and the runs since have passed.

The airline checkout was harder. The agent got from Google Flights through to an airline's passenger form, then stalled on its custom dropdowns. So the guard test finishes on a labeled fake checkout, not a real one.

Holding 60 fps on a server with no GPU meant giving up live backdrop blur for static glass. And a CI run caught a real bug on Ubuntu: under dash, the remote script's stdin watchdog killed flashd the moment it started. It only worked on my machine because my sh is bash.

Accomplishments that we're proud of

  • 57x faster than Grok Bot on the same task at the medians: 5.9 s against 336 s, with Flash's runs checked by the done gate.
  • A fare watch costs no model calls: it fired in 29.8 s with 0 LLM calls.
  • Actions are constrained. Ops are an enum and targets are numbers from a table of real elements, so the model can't invent a click target or a result.
  • About five cents a run.

What we learned

A decisions model is a better primitive than a chat model for most of what an agent does. Most steps aren't open-ended generation, they're a choice among things already on screen. And the least reliable part of an agent is the part that writes free text, because that's where it can make things up.

What's next for Flash

The goal is instant. Today the model still writes each action. Next, the Decisions API makes the choice: for every element on the page, score "is this the Search button for this form?" in one call, take the highest, and click it. No free-text action, no invented option, about a third of a second per step. Once a flow works, it compiles to a script and the next run needs no model at all.

Additionally, I want to turn Flash into a real consumer product. A personal assistant that actually does things for you, like booking, paying bills, filling forms and rescheduling, has mostly been something only executives with a human assistant get. Flash can make that available to everyone.

Built With

Share this project:

Updates

Submission history