Tabvio

• 

  Tabvio

Inspiration

I wanted to build a browser agent that could actually take work off a person's plate. A lot of everyday web tasks involve the same repetitive steps: searching, comparing options, navigating between pages, filling out forms, and preparing an action for completion. An agent should be able to handle that work autonomously instead of requiring the user to guide every click.

At the same time, there are moments that should remain with the person: entering payment information, authenticating an account, making a judgment call, or approving an important action. That became the idea behind Tabvio: give the agent the errand, let it handle the work, and bring the user back in only when their involvement is actually needed.

What it does

Tabvio is a watchable browser agent that performs multi-step web tasks and errands autonomously.

You describe a goal in plain language, and Tabvio navigates real websites, searches for information, compares options, interacts with forms, works across tabs, and continues toward the goal without requiring step-by-step instructions.

The user can watch the browser live while the agent works.

When Tabvio reaches a point that requires human participation — such as payment, authentication, missing personal information, or approval — it pauses and asks the user to step in. Once the user provides what is needed, Tabvio can continue the same task. For example, Tabvio can find dinner that matches a user's requirements, choose the items, add them to the cart, and take the order to checkout. When payment is required, it stops and hands control back to the user. The idea is simple: Tabvio handles the errand, and the human handles the moments that should remain human.

How we built it

Tabvio is built in Python using the AWS Strands Agents SDK, Amazon Bedrock, Amazon Bedrock AgentCore Browser, Playwright, and FastAPI. The Strands agent receives the user's goal and determines how to move the task forward using a set of browser capabilities. Models hosted through Amazon Bedrock power the agent's reasoning and decision-making.

Amazon Bedrock AgentCore Browser provides the managed browser environment where tasks run, while Playwright gives Tabvio the ability to navigate pages, interact with websites, fill forms, scroll, and work across tabs.

Rather than treating browser automation as a fixed script, Tabvio continually works from the current state of the website. This allows it to respond to changing pages, unexpected results, and different paths through a task. FastAPI manages the application and task lifecycle, including the live browser experience and transitions between autonomous execution and human participation. When the user needs to step in, Tabvio preserves the task so the user can interact with the browser and then hand control back to the agent without starting over.

Challenges we ran into

One challenge was making Tabvio adapt reliably as websites changed during a task. Browser tasks rarely follow one predictable path. Pages load differently, interfaces change, actions can fail, and the same goal may require a different sequence of steps depending on the website.

Another harder problem was deciding when autonomy should stop. Payment and authentication flows vary widely, and sensitive moments are not always represented by a single obvious button. Tabvio needs to complete as much of the errand as possible while recognizing when the next step should belong to the user.

This became especially important with a remotely hosted browser. The transition between agent and human needed to preserve the work already completed so the user could step in naturally and then return the task to Tabvio.

Accomplishments that we're proud of

I'm especially proud that Tabvio can take a high-level goal and turn it into real browser actions across multiple steps without requiring the user to specify every interaction.

It can navigate real websites, recover when things do not go as expected, work across tabs, interact with forms, preserve a task during human interruptions, and continue afterward.

I'm also proud of the live browser experience. Instead of an agent working invisibly in the background, the user can watch the errand happen and take control when necessary.

Most importantly, Tabvio demonstrates the interaction model I wanted from the beginning: the user delegates the work, the agent handles as much of it as it can, and control returns to the person when the task genuinely requires them.

What we learned

Building Tabvio showed me that useful autonomy is not just about how much an agent can do on it's own. It is also about knowing when the user should be involved.

Real browser tasks are dynamic. An agent has to adapt as the task progresses, handle unexpected situations, and continue making progress without requiring constant supervision.

The biggest takeaway was that human-in-the-loop works best when it is part of the product experience rather than an exception to it.

The goal is not to remove the user entirely. It is to let the agent handle the repetitive work and bring the user back when judgment, approval, browser authentication, payment, or personal input is actually needed.

What's next for Tabvio

The next step is expanding the range of everyday errands Tabvio can handle reliably across more websites.

I want to make human handoffs even smoother, improve reliability when websites behave unexpectedly, and expand the kinds of multi-step tasks people can confidently delegate.

Longer term, I want Tabvio to become something you give an outcome rather than a sequence of instructions:

"Find dinner for tonight."

"Research these options and prepare the best one."

"Fill this out for me and ask me when you need something."

You give Tabvio the errand. It handles the work and brings you back in when the task requires you.

Built With

• agents
• amazon-web-services
• fastapi
• playwright
• python
• strands