Conversational planning with hard constraints: toll avoidance, deterministic 7 AM deadline calculation, and interactive MapLibre map.
Route-aware stop discovery: Finding an on-route cafe via Overpass API, adding 15-min dwell time, and recalculating departure to 6:12 AM.

Inspiration

Modern navigation apps are fundamentally manual: drivers must guess departure times, mentally compute detour durations, and tab between search engines and map layers to locate stops along their path. Meanwhile, traditional Large Language Models (LLMs) hallucinate timestamps and struggle with spatial arithmetic.

We built RoutePilot for the Build, Ship, Shape: Amazon Developer Hackathon 2026 (Alexa+ Track) to create a true voice-first travel copilot. RoutePilot bridges natural-language conversational reasoning with deterministic geospatial execution, ensuring that users can simply state their travel goals (e.g., "Get me to Pune Airport by 7 AM, avoid tolls, and find a coffee shop on the way") and receive mathematically verified, reliable navigation plans.

What It Does

RoutePilot is a conversational navigation cockpit and Alexa+ experience simulator that solves complex multi-turn journeys:

Deadline-First Arithmetic: Instead of asking when you want to leave, RoutePilot works backward from your hard arrival deadline, computing the exact departure timestamp so you arrive on time without guesswork.
Route-Aware Stop Discovery: When you ask for a coffee stop or EV charger, RoutePilot queries OpenStreetMap POIs within a strict corridor along your computed route geometry—never searching generic city centers and never fabricating places.
Deterministic Constraint Enforcement: Hard constraints (toll avoidance, highway restrictions, deadlines) and soft preferences are evaluated in pure TypeScript outside the LLM. Verified badges (✓ avoid tolls, ✓ arrival deadline) reflect true router evaluation.
Multi-Turn Continuity & Stale-State Resilience: Adding or changing a constraint in subsequent turns preserves active stops, deadlines, and dwell times. If a provider rate-limits, RoutePilot marks the existing route stale on the map while retaining metrics, avoiding UI flaps.
On-Device Neural Voice: Runs Kokoro-82M locally in the browser via a dedicated Web Worker with WebGPU acceleration. Spoken audio is decoupled from chat Markdown, delivering natural pacing (speed: 0.95) with zero cloud audio latency or API cost.

Mathematical Formulation

To eliminate LLM arithmetic hallucinations, all travel calculations are deterministic:

$$\text{Total Duration} = \text{Driving Duration} + \sum \text{Stop Dwell Times}$$

$$\text{Latest Departure} = \text{Arrival Deadline} - \text{Total Duration} - \text{Buffer}$$

For deadline-first itineraries, the system calculates exact ISO-8601 timestamps and displays clear driving vs. dwell time breakdowns.

How We Built It

Agent Orchestration & Protocol: Built a self-hosted Model Context Protocol (MCP) server over Streamable HTTP (MCP specification $\ge$ 2025-11-25) paired with a Groq agent loop (openai/gpt-oss-120b). An Amazon Bedrock Converse integration (amazon.nova-2-lite) is also implemented behind a provider abstraction.
Deterministic Geospatial Core:
- Valhalla Routing Engine: Generates candidate routes and decodes precision-6 polyline geometries to GeoJSON.
- OpenStreetMap Nominatim: Geocodes human-readable queries to coordinates.
- OpenStreetMap Overpass API: Searches amenity corridors along the route geometry, with automatic bounded Nominatim fallback.
Multimodal Navigation Cockpit: React 19, TypeScript, Vite, and Tailwind CSS v4 featuring:
- MapLibre GL JS: Hardware-accelerated vector map rendering (OpenFreeMap Liberty tiles) with dynamic route LineStrings and linked waypoint markers.
- Speech Formatter (speechFormatter.ts): Normalizes chat Markdown, expands abbreviations (km $\rightarrow$ kilometers), and formats ISO timestamps into natural 12-hour spoken speech.
- Local Neural Speech: WebGPU-accelerated Kokoro-82M ONNX model (q8) running inside a Web Worker, with seamless browser speechSynthesis fallback.

Challenges We Faced

Token Budgeting on Dense Geometries: Returning raw GeoJSON coordinates (hundreds of lat/lon pairs) to the LLM rapidly triggered Groq's 6,000 TPM limit and HTTP 413 errors. We engineered compactToolResult, stripping raw coordinates before model ingestion (~95% token reduction) while preserving full uncompacted geometries for the MapLibre frontend.
Public API Rate Limits & Timeouts: Public Overpass API interpreter instances frequently return 429 rate limits or 504 timeouts. We implemented a 2-stage exponential backoff and a transparent bounding-box fallback to Nominatim.
Voice Realism without Cloud Dependency: Initial speech output sounded mechanical when reading Markdown lists and colons. We built a dedicated speech formatting pipeline that separates visual chat Markdown from spoken navigation instructions, tuned pacing to 0.95, and introduced selectable personas (af_bella Warm Copilot).

Accomplishments We're Proud Of

100% Green Test Suite: 150 automated backend unit, domain, and integration tests, alongside 93 Playwright E2E browser tests across desktop, tablet, and mobile viewports.
True On-Device Intelligence: Zero audio or text is sent to third-party cloud TTS engines. All speech synthesis runs locally on the client via WebGPU/WASM.
Strict Protocol Compliance: Full adherence to the Model Context Protocol Streamable HTTP standard, maintaining clean separation between reasoning and deterministic geospatial execution.

What We Learned

LLMs excel at conversational intent parsing and tool selection, but arithmetic, time offsets, and spatial checks must remain strictly deterministic.
Designing a voice-first multimodal assistant requires decoupling what the user reads on screen from what they hear aloud while driving.

What's Next for RoutePilot

Real-time live traffic integration using dynamic speed profiles.
Native Alexa+ device integration when developer preview access opens.
Multi-modal vehicle telemetry integration (EV battery state-of-charge calculation and charging corridor planning).