Relay

Inspiration

Every team has someone who does the same browser routine every morning: log into a tool, download a report, filter a spreadsheet, upload the result somewhere else. It’s tedious and easy to get wrong. Automating it usually means waiting on engineers, and traditional bots tend to break as soon as a website changes. We wanted the person who already knows the task to be able to automate it, while letting IT stay in control of what’s allowed.

What it does

Relay is a desktop app that turns repetitive browser tasks into reusable AI-assisted agents:

• Record yourself doing the task, or describe it in plain language. Relay turns it into an editable step-by-step workflow.
• Agents run in a real browser on your computer, using your own signed-in accounts.
• Before a risky step, like an upload, the agent pauses for you to check, then continues.
• Records, files and spreadsheets are saved, can be previewed, and export to CSV or Excel.
• Agents are private by default and can be shared. Teammates run shared agents with their own accounts or duplicate them to customize. Only the owner edits the original, and results stay private to whoever ran the agent.
• Admins manage who’s on each team and which agents, websites and data each team can use. They approve websites per team, and that approval is enforced both when accounts connect and when agents run. They can also upload spreadsheets as team data collections that agents can use.
• Passwords and verification codes are never recorded or sent to AI. Agents can only visit approved websites. If a site changes, the agent pauses instead of guessing.

How we built it

Desktop app: Built with Electron, React and TypeScript, with Mantine for a clean interface that works across platforms. Agent execution: Playwright records the user’s browser actions and replays them as automation. A Node.js backend runs the agents, and SQLite stores workflows and results. Agents can also search the web through SerpAPI. AI and models: Model APIs turn recordings and plain-language descriptions into editable workflows. A Hugging Face integration handles fine-tuning and deploying models; the first version trains text classifiers on team data.

Challenges we ran into

Keeping secrets out of the model: A recorded session puts passwords and verification codes on screen. We had to strip them out before anything is saved to SQLite or sent to a model API, while still letting the agent sign in with the user’s own accounts. Enforcing approved websites twice: Admin rules apply both when an account connects and when an agent runs. Every navigation, including links an agent finds through SerpAPI search, has to be checked against the team’s approved list.

Accomplishments that we're proud of

• The full loop works end to end: record or describe a task, get an editable workflow, run it in a real browser, review it, and get exportable results.
• Teams can share and duplicate agents without anyone handing over a password.
• Admins have real control: approved websites are enforced both when accounts connect and when agents run, not just listed in a settings page.
• We have an early version of training custom models from team data.

What we learned

• People trust automation more when they can see the steps and approve the risky ones.
• An agent that pauses when unsure is more useful than a fully autonomous one that guesses.
• Governance has to be part of the design from the start. Adding admin controls after the fact would have meant rebuilding the permission model.

What's next for Relay

• Budgets and spending limits for teams.
• Custom models trained on team data, building on our early text-classification version.
• Agents that make decisions based on results, not just follow fixed steps.

Built With

• electron
• huggingface
• mantine
• node.js
• playwright
• react
• serpapi
• sqlite
• typescript