Muni

Muni hears the promises you make in passing, tracks each one, and closes it only when real proof arrives from Ring, Fire TV or you. It nudges you early and asks before it acts.

Comment

Inspiration

I make small promises all day. "I'll call Amma tonight." "I'll pick up the medicine." "I'll be at the teacher meeting at 4." Many of them slip. Some I remember too late.

My phone has reminders. But a reminder only knows that it fired. It does not know if I made the call. A courier app says "delivered". That does not mean the parcel is on my porch. A model can return completed=true. That is not proof either.

So I built the opposite of a reminder app: a system that hears what I promise, watches the real world, and marks a promise done only when evidence shows it happened.

What it does

Muni is a commitment keeper for the home. You do not open an app. You do not ask it to save anything. It works in the background.

  • It hears promises. Muni listens through Bee and Alexa+. Small talk is ignored. "I'll call Amma at 7 tonight" becomes a contract with a deadline. A family member's "can you pick up the medicine?" waits for your OK. "Every Monday" becomes a recurring promise.
  • It watches for proof. A contract closes only when the evidence matches its policy: Ring camera vision for deliveries and the front door, pose tracking on Fire TV for workouts, a conversation record for calls, or your own confirmation. Your confirmation is always shown as "Confirmed by you", never as sensor proof.
  • It acts before you miss. One hour before a deadline: a quiet note. Twenty-five minutes before: an alert, but only if you are not already on it. These are real AWS EventBridge schedules.
  • It starts its own agents. When a parcel is left at the door, Muni starts a porch watch agent by itself. If someone carries the parcel toward the street, it saves the clip, a person crop and a SHA-256 manifest, and asks one question: was this family? Nothing goes to the police unless you approve the exact packet.
  • It does not accuse. You promised to bring drinks and you walk up with empty hands? You get a private "check before you go in". "Not seen" is never "forgot".
  • Fire TV Reality-Lock. You promised 15 squats before TV. The camera counts full squat cycles. 14 stays locked. 15 unlocks your evening. Half squats, frozen frames and replayed loops add nothing.
  • Alexa+. Ask "what did I promise today?" and Alexa+ answers from Muni's MCP server. Each answer shows the tool calls behind it.

Try it: https://100-49-183-59.sslip.io, press Try Muni live. No account needed. Every visitor gets a private copy of a full day.

How I built it

  • The brain: a Python FastAPI service with the official MCP Python SDK: Streamable HTTP, protocol 2025-11-25, 11 tools (compile_ambient_commitment, spawn_watchdog_subagent, evaluate_deadline_escalation, dispatch_police_evidence_dossier, bridge_ring_camera_to_firetv, plus read-only state tools).
  • Commitment compiler: rules first. It parses deadlines (tonight, by the 5th, day after tomorrow, after lunch), triggers (when I get home, before TV), recurrence, counterparty and how to verify. It refuses injection text and negations. When the rules find nothing in a sentence that still sounds like an intention ("the car needs servicing this weekend, I'm on it"), Amazon Bedrock Nova Lite reads it and returns strict JSON that I validate. Nova can propose a contract. It can never verify one, and low-confidence readings wait for the owner.
  • Ledger: AES-GCM encrypted SQLite (WAL). Append-only events with one trace ID from the spoken sentence to the screen. Terminal states never go back. A transactional outbox replicates to DynamoDB and S3 (SSE-KMS).
  • Scheduling: every T-60, T-25 and deadline job is an EventBridge Scheduler at() schedule → SQS (with a dead-letter queue) → reconciler. In a live test the three jobs fired +0.77 s, +0.43 s and +0.33 s after schedule.
  • Ring: Partner API for devices, capabilities, event history and HMAC-signed webhooks. WHEP live view is decoded on the server with aiortc. Verified on the Ring Developer Playground device (176 distinct 720p frames) and relayed into Reality-Lock.
  • Bee: live SSE stream (new-utterance, todo-created) plus todo backfill. Event shapes mirror the official bee-cli.
  • Vision: MediaPipe EfficientDet with IoU tracking and porch-zone change detection: 10/10 licensed real doorstep and CCTV clips classified correctly. MediaPipe Pose counts squat cycles.
  • Fire TV: a React Native for Vega app, run on the Vega Virtual Device, and a Fire OS APK with full D-pad navigation. An Android TV emulator runs a 33-step D-pad journey in CI on every push.
  • AWS: CloudFormation, EC2 managed through SSM (no SSH), Secrets Manager, CloudWatch.
  • Quality: 280 backend tests, 7 Playwright browser journeys, the emulator journey and a container replay run in GitHub Actions on every push.

Challenges I ran into

  • Bee needs hardware first. I could not get developer access without a Bee device or an Apple Watch running Bee, so I tested on a device borrowed from a friend. The stream event shapes are not documented, so I read them from the bee-cli source.
  • Ring tokens last 30 minutes, and a live view lasts 30–60 seconds. So Muni is event-driven, and it shows coverage gaps honestly instead of labelling old video as live.
  • Public MCP hosting. The MCP SDK rejected my public host with 421 Invalid Host header until I set allowed hosts and origins.
  • Bedrock said "Operation not allowed" on my first AWS account, with no reason. It worked on a second account.
  • "I did it" is not proof. The state machine took the most rewrites: a spoken claim moves a promise to awaiting your confirmation, never straight to done.

Accomplishments that I'm proud of

  • Nothing in Muni is marked done without evidence, and every state carries an honest source label.
  • Eleven AWS services live in production: Bedrock Nova, EventBridge Scheduler, SQS, DynamoDB, S3, KMS, Secrets Manager, CloudWatch, EC2, Systems Manager and CloudFormation.
  • Real AWS escalations that fire within one second of schedule.
  • Real Ring video, decoded on the server and counted by the same pipeline Fire TV uses.
  • One click for judges: no password, a private copy of the day for each visitor.
  • An open-source Agent Skill for Amazon's community skills collection: https://github.com/AmazonAppDev/devices-agent-skills/pull/16

What I learned

  • An evidence policy is a product feature, not a backend detail. People trust "not verified yet" more than a green tick that lies.
  • MCP fits physical-world actions well: small tools, explicit inputs, every call on record.
  • Rules first and a model second is cheap, fast and explainable. The model handles the long tail.

What's next for Muni

  • A native Alexa+ connection to the MCP server
  • Always-on Ring watching with longer-lived tokens
  • Shared households: promises made to you, each with its own proof
  • More proof types: bins at the curb, medicine taken, homework done

Built With

Share this project:

Updates

posted an update —

I wrote up how Muni works on AWS: rules first, then Amazon Bedrock Nova Lite for the promises the rules can't parse, EventBridge Scheduler for every T-60 / T-25 / deadline nudge (fired within one second of schedule), and DynamoDB, S3 and KMS for a tamper-evident evidence trail.

Read it on AWS Builder Center: https://builder.aws.com/content/3L6k7DDiQvM5IcijeZJpQ1YAS6E/a-home-ai-that-only-says-done-when-the-real-world-proves-it

Try Muni live: https://100-49-183-59.sslip.io

Log in or sign up for Devpost to join the conversation.

Submission history