Golden Guide

Thumbnail
Menu
Doggo
Information Conversation
Calling Function

Inspiration

Over 20% of Kingston, Ontario's population is 65 or older. When the QHacks 2026 theme was announced — "The Golden Age" — we didn't think of a bygone era. We thought of the people living through their own golden age right now, right here in Kingston.

We spoke to seniors in our community and heard the same story: "I know there's help out there, but I don't know where to start." Kingston has incredible programs — free dental care for seniors, subsidized transit passes, homemaking assistance, recreation funding — but they're scattered across dozens of websites, phone trees, and PDF applications, each with its own eligibility rules and office hours. For a 75-year-old with limited digital literacy, vision impairment, or no family nearby, navigating this system isn't just frustrating. It's a barrier to the help they need.

We asked: what if AI didn't just answer questions about these services — what if it actually took action? What if one sentence like "I'm 73 and I can't afford the dentist" could trigger an AI that searches services, checks eligibility across every relevant program, builds a personalized action plan, drafts an email to the right office, and offers to place a real phone call — all in a single conversation turn?

That's GoldenGuide.

What it does

GoldenGuide is an agentic AI assistant that autonomously navigates Kingston's municipal services on behalf of elderly residents. It doesn't just provide information — it takes real-world action.

A single user message can trigger a chain of 3–7 tools firing autonomously:

A user types:
"I'm a 72-year-old living alone on a fixed income. I need help with dental care and getting to my appointments."

GoldenGuide autonomously:

Searches the Kingston services knowledge base → finds CDCP, dental clinics, MFAP
Checks eligibility across every relevant program → qualifies for Canadian Dental Care Program, Municipal Fee Assistance, senior transit discount
Looks up real transit data → Route 12 to the dental clinic, next departure 10:15 AM
Generates a personalized action plan → 5 prioritized steps with phone numbers, addresses, and "what to bring" checklists
Drafts a ready-to-send email to the CDCP intake office
Offers to place a real AI-voiced phone call on the user's behalf

All of this happens in one turn, with real-time tool visualization so the user can watch the AI think — and every action requires explicit confirmation before execution.

Key capabilities

11 agentic tools in an 8-iteration autonomous loop powered by Google Gemini 3.0 Flash
Real phone calls to city offices via ElevenLabs Conversational AI + Twilio
Real emails to service providers via SendGrid
Real SMS — text your action plan to your phone or a caregiver's phone via Twilio
Document upload — photograph a confusing government letter and get a plain-English explanation with action items
Real Kingston Transit data — 31 bus routes, 809 stops from live GTFS feeds
19 enriched Kingston services with verified eligibility rules, income thresholds, and contact info
Calendar reminders — downloadable .ics files for appointment deadlines
Bilingual — full English/French support with localized system prompts
Voice input & output — speak naturally, hear responses read aloud via ElevenLabs TTS
Accessibility-first design — Atkinson Hyperlegible font, 18px/22px text toggle, 48px touch targets, high-contrast golden theme
Print / Download / SMS — export action plans in the format that works for each user

How we built it

Architecture: The Agentic Loop

GoldenGuide's brain is an 8-iteration agentic loop. Unlike a traditional chatbot that generates one response per message, our system sends the user's message to Gemini 3.0 Flash with 11 tool declarations.

Gemini autonomously decides which tools to call and in what order. Each tool executes, its results feed back into Gemini, and the loop continues until the model is satisfied it has done everything it can for the user.

No hardcoded workflows — the AI determines the optimal tool chain for each unique request.

Backend (Python + FastAPI)

Google Gemini 3.0 Flash with native function-calling for the agentic loop
11 tool declarations with full parameter schemas — 8 auto-execute, 3 action tools return preview-only objects requiring user confirmation
SSE streaming endpoint that emits tool_start and tool_done events for real-time frontend visualization
Real GTFS data — we downloaded Kingston Transit's actual GTFS feed (valid through January 2027) and built a parser that queries 809 stops and 31 routes in real-time
19-service knowledge base — we scraped and enriched every relevant Kingston senior service from cityofkingston.ca, including MFAP income thresholds, CDCP eligibility rules, and homemaking waitlist information
Tavily web search for real-time information beyond our knowledge base
Stateless architecture — no database, no auth. Conversation history travels with each request, keeping the system simple and privacy-respecting

Frontend (Next.js 16 + React + Tailwind CSS)

19 React components including 8 specialized interactive cards
(ActionPlanCard, EligibilityCard, EmailPreview, CallPreview, SmsCard, ReminderCard, DocumentExplainerCard, ServiceCard)
Custom markdown renderer in MessageBubble that parses assistant responses and routes embedded JSON blocks to the correct card component
SSE stream reader that updates tool activity UI in real-time as the agent works
Warm, accessible design system — golden (#B8860B), floral white (#FFFAF0), cornsilk (#FFF8DC), deep brown (#2C1810) — deliberately warm, not cold tech blue
Atkinson Hyperlegible font (designed specifically for low-vision readers) with Georgia/Merriweather for headings

External APIs

Google Gemini 3.0 Flash — AI backbone with function-calling
ElevenLabs — Multilingual TTS for voice output + Conversational AI for real phone calls
Twilio — phone call routing and SMS delivery
SendGrid — email delivery
Tavily — real-time web search

Data pipeline

We built scrape_kingston.py to crawl cityofkingston.ca and extract every senior-relevant service into a structured JSON knowledge base with eligibility rules, contact information, income thresholds, and application instructions.

We discovered the city had recently restructured their website URLs — old paths under /residents/community-services/ had moved to /community-supports/ — and updated everything accordingly.

Challenges we ran into

Getting the agentic loop right. The biggest challenge was making Gemini reliably chain multiple tools in a single turn. Early iterations would stop after one tool call, or call tools with malformed arguments. We iterated extensively on the system prompt — the instruction to "be proactive" and "scan ALL relevant programs, not just the one they asked about" was critical.

Real data is messy. Kingston Transit's GTFS CSV files start with a UTF-8 BOM (Byte Order Mark) that silently corrupts the first column header. Our transit tool returned empty results for hours before we traced it to an encoding issue and switched to utf-8-sig. The City of Kingston also restructured their entire website mid-development, breaking every URL we had scraped.

Action tool safety. We needed the AI to offer to call, email, and text — but never execute without explicit user consent. We solved this by splitting action tools into preview-only lambdas in the agent loop (returning requires_confirmation: true) with separate execution endpoints that only fire when the user clicks "Send" or "Place Call."

Streaming tool visualization. Showing tools firing in real-time required a custom SSE protocol. We emit tool_start and tool_done events from the backend generator, parse them in a streaming reader on the frontend, and update React state to show animated checkmarks — all while the agent loop is still running.

Accessibility at every level. We didn't want accessibility as an afterthought. Choosing Atkinson Hyperlegible as the base font, setting 18px as the minimum, ensuring every button hits 48px touch targets, and making voice input/output work in both English and French touched every component.

Accomplishments that we're proud of

It actually works end-to-end. A user can say one sentence and GoldenGuide will autonomously search services, check eligibility, look up real bus schedules, generate an action plan, draft an email, and offer to place a real phone call — all in one turn, with real data, real APIs, and real-time visualization.

The agentic behavior is genuinely autonomous. We don't hardcode tool chains. The AI decides to call 3, 5, or 7 tools depending on complexity. It surfaces programs users didn't know existed.

Real Kingston data. 19 verified services with actual income thresholds (MFAP: $34,793 for a single-person household). 31 real bus routes with 809 real stops. Real phone numbers.

The confirmation-before-action pattern. GoldenGuide can place real phone calls and send real emails — but it always shows you what it will say first, lets you edit it, and waits for explicit consent.

Bilingual in 30 lines of code. Switching from English to French regenerates the entire system prompt, all UI text, and all quick-topic cards without duplicating component logic.

The design feels warm. The golden-and-cream palette, serif headings, and large accessible text create something that feels trustworthy and human.

What we learned

Agentic AI is a fundamentally different paradigm. The jump from "AI that answers questions" to "AI that takes action" is enormous. The system prompt isn't just tone — it's strategy.

Accessibility is architecture, not decoration. We couldn't bolt on accessibility at the end. Font size, touch targets, and voice I/O shaped the entire product.

Real data humbles you. BOM-encoded CSVs, restructured URLs, inconsistent eligibility criteria — real Kingston data forced robustness.

SSE streaming unlocks trust. When users can watch the AI search, check eligibility, and build plans in real time, trust increases.

Kingston has incredible services that most people don't know about. Programs like MFAP unlock discounted transit, free dental, vision care, recreation funding, and food market discounts — all from a single application.

What's next for GoldenGuide

City of Kingston pilot. GoldenGuide was built with real Kingston data because we want to deploy it. With the City of Kingston as a QHacks sponsor, we're actively exploring a pilot partnership.

Expanded knowledge base. Our current 19 services only scratch the surface. Kingston has hundreds of programs across provincial, federal, and nonprofit providers.

Multi-city expansion. The architecture is city-agnostic. With a new JSON knowledge base and GTFS feed, GoldenGuide could serve Ottawa, Toronto, or any Canadian city.

Caregiver dashboard. A caregiver-facing dashboard to receive summaries, track follow-ups, and enable remote assistance would multiply impact.

Offline-first PWA. Many seniors have unreliable internet. Packaging GoldenGuide as a Progressive Web App with offline caching would make it usable even without connectivity.