Duly

Duly turns Alexa+ into your household paperwork manager: bills, deadlines, warranties and letters, handled by voice through an MCP server.

Comment

Inspiration

To be completely honest, it started at home. My dad gets a letter and asks me: "What is this? Where does it come from? What do I have to do?" Then I thought about the bigger picture: in places like the US, many households are buried in bills, letters, receipts, fines and renewals, and the paperwork can be overwhelming. A tool where you just take a photo, upload it, and get it explained, stored, and ready to answer questions or draft a reply could be a real relief for those people. For my parents it's a nice help; for someone facing a pile of paperwork every month, I think it could become something they rely on every day, once it's fully polished and ready for the public.

What it does

Duly turns Alexa+ into a household paperwork manager. You give it a bill, a fine, a warranty or a renewal notice (a photo, a PDF or text), and Duly remembers it. From then on you can just ask:

  • "What do I need to pay this month?" Every deadline with an amount, soonest first.
  • "Is my Brightwave bill higher than usual?" It compares the latest bill with earlier ones and tells you whether you used more or the price went up.
  • "Write a reply to Brightwave Energy disputing my electricity bill." A short letter with the real amounts, dates and account number; one tap makes it firmer.
  • "What happens if I ignore the parking notice?" A plain-language explanation and the next step.
  • "Is my dishwasher still under warranty?" End date, days left and how to claim.
  • "When does the car insurance renew?" The answer about that exact item, or a clear "I couldn't find it" (never an answer about another item).
  • "Give me my weekly brief." What's due, renewals you can still cancel, warranties about to end and bills above usual.

Duly keeps state across sessions: a letter you upload today shows up tomorrow in your deadlines, your weekly brief and the replies it drafts.

How I built it

Duly is a standard MCP server (spec 2025-11-25, Streamable HTTP) with eight tools, so Alexa+ or any other agent can use it. Every tool returns a short sentence meant to be spoken plus structured data for screens.

  • Amazon Bedrock (Nova 2 Lite) reads photos and PDFs and extracts issuer, amounts, deadlines and action items as validated JSON; it also writes explanations and reply letters.
  • Amazon DynamoDB stores the documents and Amazon S3 keeps the original files (private bucket).
  • Amazon Polly is Duly's voice in the web demo.
  • The web demo simulates an Echo Show: a Bedrock agent acts as an MCP client of Duly's own endpoint, the same way Alexa+ would.

The key design decision: the model only picks the tool; what Duly says comes from the tool itself. Numbers, dates and confirmations are computed in code, so Duly can't invent figures or claim actions it didn't take. Letters are checked before they're shown (no figures that aren't in the document, the right labels on reference numbers, distinct polite and firm tones) and fall back to a safe template if a check fails.

The repo has unit tests and a smoke test that asks production 17 real questions through Bedrock and checks which tool answered each one.

Challenges I ran into

  • Bedrock was blocked for hours on my new AWS account ("your account is currently being verified"). I kept building with a Gemini fallback behind Duly's LLM adapter and switched back once the account was verified.
  • Nova 2 Lite doesn't support Bedrock structured outputs, so I used a forced tool call whose input schema is the output format, validated with zod.
  • A tool result that mixes text and JSON blocks makes Bedrock silently drop the tools from the prompt: no error, Nova just answers in text. I turned the fix into an open-source library, mcp-bedrock-bridge.
  • Questions about one specific item ("when does the car insurance renew?") were answered about the nearest deadline of another item. I added an item filter with strict matching and tests over every sample document.

All of these are documented in the friction log.

Accomplishments that I'm proud of

The result itself. There isn't one part I'm prouder of than the rest. It's the fact that I can share a link and Duly is there, managing your paperwork.

What I learned

A classmate, who is also taking part in this hackathon with his own project, told me he had built a project with the help of AI. That same afternoon I started researching what I could do. I already use AI every day: to study in my software development course (DAM), as a tutor for my subjects, and for small projects. But I had never used it for something this big. Duly is my first project and my first GitHub repository. I had watched videos about using AI efficiently, and I wanted to go beyond "vibe coding", where you just talk to the app and hope for the best. I wanted to work with agents that do their job well, in an orderly way, with procedures, standards and an established methodology. So I directed the project by orchestrating AI tools: one model planned and reviewed, Claude Code implemented each task in plan mode, and I made the decisions and approved every step. That's what I learned, and I still have a huge amount left to learn.

What's next for Duly

Running it on a real Alexa+ device, the one I have at home, and seeing how it works there. The path is Amazon's Alexa+ MCP Toolkit, which is in preview for select partners today; when it opens to developers, the next technical steps are adding OAuth 2.1 to the MCP endpoint and keeping tool responses within Alexa's latency budget. Seeing Duly work on a real Alexa+ would be amazing.

Built With

Share this project:

Updates

Submission history