kanari

Track stress before you crash. Kanari uses Gemini voice check-ins and on-device acoustic analysis to forecast burnout risk and schedule recovery. Private, browser-based wellness.

Comment

Inspiration

You know the feeling: 8 hours of studying, skipped meals, and still falling behind.

As community college STEM students living alone, we saw how burnout builds quietly through scrolling loops, context switching, and procrastination cycles. Those energy leaks are hard to spot in real time. We built kanari to make those patterns visible early, so recovery can happen before a crash.

What it does

kanari is a browser-based burnout risk companion that uses short daily voice check-ins.

  • Voice check-in with Gemini Live: a natural 30-60 second conversation.
  • On-device acoustic biomarkers: local feature extraction for pauses, pitch variability, speech dynamics, and spectral signals.
  • Gemini 3 analysis layer: semantic interpretation, personalized synthesis, targeted recovery suggestions, and achievement generation.
  • 3-7 day burnout forecast: heuristic short-horizon trend guidance with confidence and contributing factors.
  • Action loop: suggestions become scheduled recovery blocks in kanari’s built-in local calendar, including recurring schedules.
  • Progress layer: check-in history, journal support, streaks, and achievements to reinforce daily consistency.

How we built it

We built kanari with Next.js App Router, React, TypeScript, Tailwind CSS, and Dexie (IndexedDB).

Gemini architecture

  • Gemini Live (WebSocket) powers real-time voice conversation during check-ins.
  • Gemini 3 Flash (REST) powers deeper reasoning and generation across multiple endpoints:
    • check-in context summary generation from prior sessions and trend signals
    • post-check-in synthesis (narrative, evidence-linked insights, suggestions)
    • semantic audio interpretation
    • suggestion generation and diff-aware refresh
    • daily achievements generation
    • optional coach-avatar recipe selection (style + seed)

Voice and ML pipeline

  • Browser audio capture with Web Audio + VAD + Meyda feature extraction.
  • Session-level acoustic scoring and quality checks.
  • Semantic + acoustic biomarker fusion.
  • Trend-based forecasting logic for 3-7 day risk guidance.

Product and UX system

  • Local-first persistence in IndexedDB.
  • In-app scheduling and recovery block management.
  • Guided onboarding with voice selection, coach avatar, and accountability preferences.
  • Appearance customization (accent color, font selection, graphics quality).
  • Feature tour and demo workspace seeding for judge-friendly exploration.

Challenges we ran into

  • Realtime synchronization between live conversation events and local acoustic timelines.
  • Browser performance tuning so signal processing stayed smooth on consumer laptops.
  • Structured output reliability across multiple Gemini 3 JSON workflows.
  • Conversation robustness under interruption, reconnect, and session-resume scenarios.
  • UX clarity for presenting complex wellness signals in a calm, actionable interface.

Accomplishments that we're proud of

  • We shipped an end-to-end loop: voice check-in -> synthesis -> forecast -> scheduled recovery action.
  • We integrated Gemini 3 across core product workflows, not a single isolated feature.
  • We implemented local-first acoustic analysis and storage for strong user data control.
  • We built resilient realtime voice behavior with barge-in handling, reconnect, and session preservation.
  • We delivered polished product depth beyond core ML, including achievements, feature tour, demo mode, and personalization.

What we learned

Voice carries strong stress and fatigue cues, and those cues become much more useful when combined with semantic context from Gemini 3.

We also learned that voice AI products need equal investment in model integration and systems reliability. Structured outputs, recovery paths, and clear UX language directly shape user trust.

What's next for kanari

  • Extend forecasting from short horizon to long-term seasonal pattern tracking.
  • Improve explainability so users can see why risk and confidence changed.
  • Add optional multimodal inputs such as wearable biomarkers (for tri-modal fusion: audio + text + biometrics).
  • Expand recovery planning intelligence with stronger adherence feedback loops.
  • Build Team Pulse for privacy-preserving aggregate wellness insight at group level.

Built With

  • gemini-2.5-flash-native
  • gemini-3.0-flash
  • gemini-live
  • indexeddb
  • next.js
  • oauth
  • react
Share this project:

Updates