BunqMate

Home screen /budget overview
destination / event suggestions
Suggestions: things to do in your budget
Image to price extraction and conversion
image to price conversion and fee breakdown
audio input
audio to currency with fee breakdown
menu upload
menu translation and conversion

Inspiration

Traveling is great; doing currency math is not. Is that dinner actually cheap? Does this souvenir fit the budget? We were tired of fumbling with Google for rough exchange rates only to get surprised by hidden fees later. We wanted the bottom line: What exactly is this going to cost my bunq account right now?

What it does

BunqMate is your ultimate multimodal travel finance companion. Point your camera at a price tag, snap a full restaurant menu, or get the vendor to say a price out loud. We instantly extract the amount and convert it to EUR using bunq's exact real-world fee models (ZeroFX for card, Wise for transfers).

It doesn't stop at conversions. BunqMate tracks your trip budget in real-time, displays your live bunq balance, and serves up AI-driven, budget-aware local recommendations to keep your spending on track.

How we built it

Frontend & Backend: Fast, responsive SPA served by a FastAPI backend.
Visual AI: Claude Haiku for lightning-fast price and menu scanning, seamlessly escalating to Claude Sonnet for tricky or low-confidence images.
Voice AI: Audio → S3 → Amazon Transcribe (40+ auto-detected languages) → Amazon Nova Lite on Bedrock to instantly pull structured monetary values.
FX Engine: Mastercard API for card rates + 0.5% ZeroFX fee; Wise API for transfers + ~0.65% fee, with an ECB/Frankfurter fallback.
Geolocation: IP-based currency resolution (because a "$" in Canada isn't a "$" in the US).
bunq Integration: Full 3-step RSA auth flow, SQLite for local tagging, and live balance sync.
Smart Recs: Google Maps + Ticketmaster + Claude for context, time, and budget-aware local suggestions.

Challenges we ran into

Mastercard API Auth: Wrestling with OAuth 1.0a and RSA-SHA256 signatures was a beast, but we nailed the scheme.
The "$" Dilemma: Resolving ambiguous currency symbols. We combined geolocation with Claude's confidence scoring to reliably crack edge cases.
Multilingual Audio: Accurately transcribing spoken prices in 40+ languages—and structuring that data—required a complex, multi-stage pipeline.

Accomplishments we're proud of

True Cost, Zero Guesswork: Snapping a photo and instantly seeing the exact EUR cost (fees included). No estimates.
Menu Magic: Scanning an entire foreign menu and instantly getting English translations paired with exact EUR prices.
Pinpoint Accuracy: Engineering a fee model that perfectly mirrors bunq's ZeroFX and Wise paths.
Universal Voice: Flawless voice input across 40+ languages without the user ever needing to select a language.
Budget-Smart Recs: AI local suggestions that actually care about how much money you have left to spend.

What we learned

Multimodal AI is Magic: LLMs can read handwritten, smudged, or weirdly angled price tags with shocking accuracy.
Divide & Conquer: Reliable voice input requires treating transcription and data extraction as two completely separate jobs.
Accuracy Matters: The gap between the "mid-market rate" and "what you actually pay" is a massive factor in building user trust.

What's next for BunqMate

Mobile app: Moving from web-app to a Bunq-ecosystem integrated mobile app.
Real-time alerts: Push notifications when you're nearing your trip budget.
Auto-tagging: Parsing bunq transaction descriptions to organize payments effortlessly.
Group Mode: Shared trip budgets for friends traveling together.
Ecosystem Growth: Expanding payment method coverage as bunq drops new products.