Verity - Trustworthy Household Action Agent

A context-aware household agent for Alexa+ that uses MCP tools to coordinate simulated rides, deliveries, and home activities with explicit human approval before consequential actions and action logs.

Comment

Verity: a household agent that asks before it acts

Where it started

Some of the things I wanted from a home assistant were small: get my mother a ride to the market, order milk and bread, check whether my father is home before sending someone to help. Each one is a short sentence to say, but in practice each one means opening several apps, repeating names and places, and comparing options. Often the person who is out isn't there to answer, and the assistant can't tell.

I kept asking what it would take for these to be a single spoken request: "Book Mother a ride to the market." The assistant should know who is speaking, who the request is for, whether that person is home, what the cheapest and fastest options are, and when it has to stop and ask. I didn't want a voice command to book something without a check. I wanted it to coordinate the work and then let a person decide.

That's Verity. It is a household agent that turns a spoken request into a set of checks, a comparison, and a decision card that someone has to accept or decline.

What the household sees

Verity runs as a small house on screen, with Mother, Father, Son and a Housebot robot moving between rooms. Each request opens its own popup with the steps it took in plain words. When it needs a decision, the decision appears in that popup, as a card that says what is being proposed and why.

A few things happen the way they would in a real house:

  • Rides. "Book Father a ride to the office" compares three simulated rides on price, time, the distance from the house, and how far the driver is. The choice depends on the rider's saved preference. Someone who wants the cheapest ride is never moved to a dearer one. Someone who wants the fastest is moved only when the extra fare per minute saved is one they said they'd accept. Verity asks "cheapest or fastest?" once, and remembers the answer for that person.
  • Waiting for a ride. Mother's ride shows a countdown on her chip, and she comes back home when it reaches zero. Nobody has to remember to update the house.
  • Presence. Someone who says "I'm going out" is marked away. A request for someone who is out is refused with the time they're back, and nothing is changed.
  • Shared TV. When Father starts the TV, everyone at home is asked whether they want to join. Saying no sends them to their own room.
  • Groceries. Order milk and bread compares simulated delivery providers, proposes the cheapest, and orders only after Accept.
  • Housebot. "Massage me" means the speaker. Housebot can't help someone who is out.

Why every action waits for a person

The principle I kept returning to was that the AI can coordinate the work, but people stay in control of anything that costs money or changes the house. So a ride or an order is always a proposal first. The model is not given the tools that approve or book, and a call to one is refused. Only the Accept button, acting on one specific proposal, can finish it. Two people's requests can wait at the same time, and each is decided on its own card.

I learned this the hard way. In one early test, the model approved and booked a ride on its own, because it had been given the approve tool. That gap is closed now, and the test is recorded.

How it's built

Verity is a real, self-hosted MCP server over Streamable HTTP, with explicit tools for the home, rides, orders, presence and robot. Python services enforce the rules: ranking, approvals, presence and the refusal cases. The browser page is the simulated Alexa+ surface; the agent host sends each request to the model, or to a labelled scripted path when no key is set. People and preferences are in a local SQLite database, and the seed state is copied for every test run, so the tests never change a running house.

The model can be Groq, OpenAI, or Amazon Bedrock through its Converse API. A live call to the Nova model on Bedrock returned successfully. Several full Verity requests also ran through the Bedrock tool loop on a test copy of the house: a TV start, a ride proposal, and a booking attempt that the approval gate held back. The trace is in the repository. The live demo currently runs on Amazon Bedrock (Nova). Groq is also supported.

What was hard

  • Timers that wouldn't stop. A rider's countdown kept resetting to three minutes, because the server refused to mark them away without a clock time, and the host kept asking again. I reproduced it with the real house tools, then moved the countdown to the server, which now owns it.
  • Requests that stacked. Once each request had its own popup, popups multiplied. Finished ones now close when more than three are open, and the ones still working or waiting stay.
  • Names and turns. Speaking as "Father" to mean the father, and "me" to mean the speaker, both needed explicit rules, on the scripted path and the model path.
  • Model limits. The model sometimes sends malformed tool calls, and a daily token limit on the account blocked real answers for a while. Both are handled now: one retry for a malformed call, and a clear message for the limit.

What I'm proud of

Every decision the household can make in Verity goes through one card, and every card can be traced back to the tool results that produced it. A declined ride says which ride it declined, and nothing is booked by mistake. The tests run against a copy of the house, so the demo can't be broken by testing it.

What I learned

A useful action agent is less about connecting a model to APIs and more about deciding where the model stops. Verity is the model choosing tools, and software enforcing the rules that matter. The second thing I learned is that a demo has to be honest about what is simulated. Verity's rides, prices, deliveries and robot are simulated, and it doesn't connect to Alexa+ or a physical speaker. Saying that plainly is part of being trustworthy.

What's next

Real providers and smart-home devices, stronger identity so that a voice can be tied to a household member, a persistent audit history, and a voice channel that uses real speech recognition. Those are the steps between this prototype and a household that really uses it.

Verity: everyday household actions, coordinated through a sentence, and decided by the people they affect.

Built With

Share this project:

Updates

Submission history