VETC Buddy — AI Trip Planner for Vietnamese Drivers

Inspiration

Vietnamese drivers juggle Google Maps, toll price forums, weather apps, and mental math just to answer one question: "How much will this trip cost me?" We built an AI co-pilot that answers it in one sentence — in Vietnamese, with real data, through voice or text. Qwen's strong Vietnamese language support and vision capabilities made it the natural choice.

What it does

Say "Lên kế hoạch đi Đà Lạt từ HCM" and the AI chains up to 8 tools automatically: route on an interactive map, gate-by-gate toll breakdown, fuel cost, trip summary, weather forecast along the way, rest stops every ~150km, and a wallet balance check — all from one message.

• 16 AI tools — routing, toll estimation (exact data for 13+ expressways), fuel cost, weather, POI search, wallet check, trip history, web search, image analysis
• Voice-first — Vietnamese speech recognition with push-to-talk and auto-silence detection
• Vision — snap a toll receipt and Qwen-VL-Max extracts station name, fee, plate number, timestamp
• Route comparison — compare up to 3 alternatives side-by-side with cost breakdowns
• Multi-stop trips — per-leg cost tracking for 2–8 stops

How we built it

• Qwen 3.5 Flash for chat + tool calling, Qwen-VL-Max for image analysis, both via DashScope
• System prompt-driven tool chaining — a trip query triggers route → toll → fuel → summary automatically, extending to weather + rest stops + wallet for longer trips
• Free geo stack — OSRM (routing), Nominatim (geocoding), Overpass (POIs), Open-Meteo (weather). Zero paid geo APIs
• Curated Vietnamese toll data — gate-by-gate fees for 13+ expressways with vehicle type multipliers
• Next.js 16 + MapLibre GL v5 (OpenFreeMap tiles) + Vercel AI SDK for streaming chat with rich tool-result cards
• Web Speech API for Vietnamese voice input; responsive layout (desktop sidebar + mobile bottom sheet)

Challenges we ran into

• Tool chaining reliability — getting Qwen to consistently call the right tools in the right order took many system prompt iterations
• Routing through Laos — OSRM sometimes routes HCM→Ha Noi through Laos; we added corridor waypoints to keep routes on Vietnamese expressways
• No toll price API exists — we manually researched and curated gate-by-gate toll data for 13+ expressways
• Vietnamese voice quirks — Web Speech API support varies across browsers; we built auto-restart logic and 14 error codes to handle edge cases
• Map-chat sync — keeping the map in sync with tool results required a dedicated hook to parse all messages and derive map state

Accomplishments that we're proud of

• One sentence, eight tools — "plan a trip to Da Lat" returns route + toll gates + fuel + summary + weather at 5 points + rest stops + fuel stations + wallet check
• Zero paid APIs for geo/maps/weather — only Qwen via DashScope
• Vietnamese-first — not translated from English; every prompt, label, and advisory is written for Vietnamese drivers
• Useful vision — toll receipt extraction solves a real workflow (reconciling receipts with VETC accounts)

What we learned

• System prompts are the product — prescribing tool-call sequences and teaching the model to pass values between tools is where 80% of reliability comes from
• Qwen handles Vietnamese well — diacritics, relative dates ("thứ 7 này"), place names all work naturally
• Open-source geo is production-ready — OSRM, Nominatim, Overpass, Open-Meteo are limited by rate limits, not quality
• Tool calling transforms chatbots into agents — 16 tools with structured outputs turn vague answers into real calculations

What's next

• Real VETC integration — replace mock wallet/history with actual VETC APIs for live balance, transactions, and top-ups
• Expanded vision — toll gate identification, vehicle damage assessment, traffic sign translation for foreign drivers
• Toll data crowdsourcing — let drivers verify and update toll prices to cover more than 13 expressways
• Trip sharing — export plans as shareable cards or PDFs
• Offline mode — cache routes and toll data for areas with poor connectivity

Built With

• nextjs
• qwen