SpendWise Voice

Inspiration

Most people forget to log expenses, and opening a budgeting app feels like a chore. By month end, nobody knows where the money went. Typing is also hard for elderly users, and most finance apps work only in English. We wanted expense tracking to feel like talking to a helpful person, in the language people actually speak at home: English, Urdu, Hindi or a mix.

What it does

SpendWise Voice lets users manage personal finances by conversation: Add expenses by voice or text (“500 rupay petrol pe kharch kiye”, “kal 300 lunch”) Totals by day, week, month or any date range, and the top spending category Month-over-month comparison and a spoken weekly summary Monthly budgets with warnings at 80% and when exceeded, plus a “budget left” check Edit or delete past expenses, always with a confirmation before deleting Recurring bills such as rent and internet: at the start of the month the assistant reminds the user, and adds the bills only after the user says yes. If the user doesn’t answer, it keeps reminding. The user can skip a month or remove a bill Multi-step sentences like “1000 was wrong, I spent 500” A live dashboard of spending by category

How we built it

A Python MCP server (Streamable HTTP) exposes 19 tools: expenses, budgets, analytics, recurring bills and summaries, stored in SQLite A simulated Alexa+ experience is a web app (Starlette + JavaScript) with a mic button and spoken replies. It turns the user’s sentence into MCP tool calls using the Gemini API, with retries and a backup model Each reply shows which MCP tool ran, so the tool layer is visible in the demo We followed the Alexa+ customer experience principles: tasks finish by voice, actions are confirmed, minimal data is used, and errors always give a clear next step Important note: The Alexa+ experience is simulated. In real Alexa+, language understanding is handled by Alexa+ itself and our MCP server only exposes the tools. The Gemini layer exists only in the simulator.

Challenges we ran into

Our first rule-based language parser failed on Urdu/Hinglish sentences, so we replaced it with Gemini The MCP Python SDK v2 renamed core APIs mid-project, so we pinned mcp<2 Gemini 503 and timeout errors during testing led us to add retries and a backup model Keeping the model from taking unsafe shortcuts, like adding an expense and asking a question in the same reply, or claiming something was done when no tool ran Real Alexa+ MCP onboarding needs OAuth 2.1 account linking and partner access, which were not available to us as a solo developer

Accomplishments that we're proud of

A voice-first tracker that understands English, Urdu, Hindi and Roman Urdu A recurring-bill flow that asks permission instead of silently changing the user’s data 19 well-defined MCP tools that any MCP-compatible assistant could use Built end to end as a solo developer in a few days

What we learned

How MCP tools are designed and why confirmation steps matter for destructive actions Voice replies must be short, clear and forgiving Language understanding is best left to a model, while money logic should stay deterministic and testable

What's next for SpendWise Voice

Deploy as a real Alexa+ add-on with OAuth 2.1 account linking and per-user data Proactive morning reminders through Alexa+ routines or notifications Cloud database and AI-generated insights Savings goals, family shared budgets and multiple voice profiles Receipt scanning and bank SMS import Echo Show visuals using MCP Apps

Built With

• css3
• gemini-api
• html5
• javascript
• mcp
• python
• sqlite
• starlette
• uvicorn