HARMONY

What it does

HARMONY is a voice-based financial and payment AI agent. It listens to user requests, identifies paid actions, explains costs in MNEE stablecoins, analyzes the user’s past spending data (simulated), and provides contextual financial advice. With explicit user approval, HARMONY simulates agent-assisted payments, ensuring human-in-the-loop financial control.

How we built it

Built a conversational AI agent using LiveKit

Used Deepgram for speech-to-text

Integrated an LLM (GPT-4o-mini via OpenRouter) for financial reasoning

Implemented spending memory to simulate user transaction history

Designed a payment flow that simulates MNEE stablecoin transactions

Prioritized agent intelligence and safety over UI complexity

Challenges we ran into

Handling voice input reliably during rapid prototyping

Managing agent responses without full text-to-speech support

Designing meaningful financial advice without real user data

Ensuring the agent assists decisions rather than acting autonomously

We addressed these by focusing on text-based stability, simulated data, and clear financial guardrails.

Accomplishments that we're proud of

Built a working voice-enabled financial AI agent

Implemented spending-aware financial advice

Demonstrated human-approved agent payments

Aligned closely with the MNEE programmable money vision

Created a clean separation between agent intelligence and interface

What we learned

Financial context is critical for responsible AI agents

Stablecoins are better suited for agent-driven workflows than traditional banking

Human-in-the-loop design builds trust

Simplicity matters more than over-automation in early systems

What's next for HARMONY

Browser-based overlay interface (Grammarly-style)

Real-time MNEE on-chain transactions

Stronger budgeting and risk analysis

Multi-agent payment coordination

Deployment as a shared financial AI service

Built With

• deepgrappython
• livekit
• sqlite