Hotfix

Inspiration

Most back-office work in logistics, healthcare, and local government runs on websites built twenty years ago. These portals don't have public APIs, they don't have webhooks, and the organizations running them aren't building new ones anytime soon. Companies literally pay people to sit at a desk, read spreadsheets, type numbers into clunky forms, and copy results back into their databases.

Autonomous browser agents were supposed to fix this, but making an AI browse a website for every single request is a terrible idea in production. Having an LLM inspect thousands of HTML elements takes 20 seconds and burns real money on tokens every run. On the other hand, traditional scripts like Playwright run in 300 milliseconds for free, but they break the second someone renames a CSS class.

We built Hotfix to combine the best of both worlds. It turns old, API-less websites into fast REST endpoints that run deterministically in milliseconds, while using an AI agent strictly as an on-call mechanic that steps in to fix broken selectors when the site updates.

How We Built It

We split the system into two parts: a high-speed execution path and an automated repair loop.

The Fast Path: A lightweight Python runner drives Playwright using a simple selector map (contracts/selectors.json). When an API call comes in, it doesn't touch an AI model at all. It just fills the inputs, clicks the button, and hands back clean JSON in about 300 milliseconds.
Sentry Catching Failures: We wrapped our FastAPI backend and runner in Sentry. We killed Playwright's annoying 30-second default timeout and dropped it to two seconds. If a button isn't where it should be, the runner fails immediately, dumps a screenshot and a snippet of the broken DOM, and sends the crash data to Sentry.
The Browser Use Healer: When the backend catches that failure, it wakes up a Browser Use agent. The agent launches the browser, looks at the new layout, figures out how to complete the action, and identifies the updated selector. It patches contracts/selectors.json, logs the self-healing event in Sentry, and replays the original request so the user still gets their data. From Sentry, we used Tracing and AI Agent Monitoring.
The Dashboard: We built a quick interface with React, Vite, and Paper components to show the system working. It gives you a live look at the API, tracks the massive speed and cost difference between cached runs and agent runs, and shows a side-by-side diff of the old versus new selectors.

Conceptually, the economics make complete sense. Sites only change their layout once every few months. Running a deterministic script 99% of the time costs zero dollars and takes under half a second. Paying a few cents for an agent only when something actually breaks keeps your operational costs near zero while keeping the integration alive.

Challenges We Faced

Our biggest headache early on was browser context isolation. In our test portal, we built a Chaos Mode toggle to scramble HTML IDs and wrap buttons inside modals. When we tested it manually in Chrome, it worked fine. But when Playwright launched headlessly, it opened a totally fresh session that didn't see the mutated DOM at all, so our tests kept passing when they were supposed to fail. We fixed it by keeping the chaos state in server memory on Python so every new headless browser saw the scrambled markup.

We also ran into issues grabbing DOM snippets during crashes. When a selector like #submit-btn disappeared, our error handler initially tried to inspect that specific element, which just threw another timeout error and crashed the script before saving anything. We had to widen the snapshot to grab the parent form container instead, trimming it down so the AI got enough context without hitting token limits.

Lastly, coordinating three people using AI coding tools can get messy fast. We avoided merge conflicts by locking our JSON schemas and file paths before writing any code. Everyone worked in their own folder and tested against static mock files so nobody was blocked waiting on someone else's backend.

What We Learned

AI browser agents shouldn't be handling routine tasks that deterministic code can do in 300 milliseconds. They belong in the maintenance loop.

Using Browser Use as an automated site reliability engineer gives you the raw speed of traditional scripts with the problem-solving ability of an AI model when things change. We also realized how useful observability tools like Sentry are when you connect them to agents. Instead of Sentry just being a dashboard where error alerts sit around, feeding those traces and DOM snapshots straight into an agent turns passive crash alerts into code that fixes itself.