Inspiration
My dad’s Parkinson’s inspired Noongil. For many people with Parkinson’s, the problem is not a lack of apps. The problem is that using a smartphone itself can become exhausting. Tremor, stiffness, fatigue, and low-energy days can make tapping, typing, and navigating screens difficult. I wanted to build something that feels more humane and accessible: a voice-first companion that can listen naturally, help with daily check-ins, and connect the member, caregiver, and doctor through one continuous care loop.
What it does
Noongil is a voice-first wellness companion built around real-time Gemini Live interaction.
A member can speak naturally with Noongil instead of filling out forms. The app guides daily wellness check-ins about mood, sleep, energy, symptoms, and medication adherence. Those check-ins become structured data instead of disappearing as transcripts.
On the caregiver side, Noongil provides a dashboard that shows:
- wellness and symptom trends
- missing check-in days
- a heuristic symptom relationship graph across days
- care plan controls like reminders
- voice messages back to the member
The graph is not diagnosis. It is a heuristic pattern engine that helps surface likely contributing factors, such as poor sleep, missed medication, or stress triggers before a worse symptom day.
Noongil can also pair with Ray-Ban Meta smart glasses as an optional hands-free multimodal context surface.
How we built it
We built Noongil as a full-stack care workflow across mobile, backend, and caregiver surfaces.
On iPhone, the app is written in Swift and SwiftUI. The member experience uses Gemini Live for low-latency, interruptible voice interaction. Firebase Auth and Firestore handle identity, syncing, and app state. The app supports caregiver-linked workflows, reminders, and voice message playback.
On the backend, we built a Fastify + TypeScript service deployed on Google Cloud Run. The backend handles caregiver APIs, dashboard serving, graph ingestion, reminders, and report-oriented workflows. Firestore stores structured check-ins and caregiver data. Neo4j stores the longitudinal graph of symptoms, sleep, medication adherence, triggers, and derived relationships. We also built backend service paths for Vertex AI and cloud-side orchestration.
For the caregiver dashboard, we built a dedicated frontend app focused on four areas: check-ins, causal engine, care plan controls, and messages.
Challenges we ran into
The hardest part was turning a natural live voice conversation into reliable structured health data.
We had to solve several practical issues:
- making live voice interaction feel responsive without becoming robotic
- ensuring important check-in topics were captured consistently
- synchronizing the member app, Firestore, backend graph ingestion, and caregiver dashboard
- handling date and timezone edge cases so a late-evening local check-in did not appear on the wrong day in the graph
- making the caregiver dashboard explain “why today got worse” without implying diagnosis
- balancing demo speed with production-safe architecture, especially for auth, deployment, and public-repo hygiene
Another challenge was building something that is both technically ambitious and emotionally grounded. We did not want a generic chatbot. We wanted a tool that would genuinely reduce friction for someone having a hard day.
Last but not least, getting the app working with smart glass such as Meta's Ray Ban was tricky due to lack of their development kit documentations, handling Gemini Live web sockets. But eventually, we got it working to a point where people are asking if it is like a Baymax!
Accomplishments that we're proud of
We’re proud that Noongil is not just a concept. It is a working care loop.
We built:
- a real-time voice-first member experience
- a caregiver invitation and sign-in flow
- a caregiver dashboard with trends, missing days, and heuristic causal graphing
- reminder controls that flow back into the app
- voice messages from caregiver to member
- a deployed Google Cloud backend
- a longitudinal graph model that turns check-ins into more useful caregiver insight
We are especially proud that the project stayed focused on accessibility and dignity, instead of adding AI for its own sake.
What we learned
We learned that the hard part of voice agents is not just speech. It is structure, continuity, and trust.
A live model can produce a great conversation, but care workflows require much more:
- consistent extraction of meaningful fields
- resilient syncing between mobile and backend
- explainable heuristics instead of opaque conclusions
- interfaces that work well for both non-technical caregivers and members with limited energy or motor control
We also learned a lot about integrating real-time Gemini interaction with Google Cloud services and about where fast hackathon architecture differs from long-term production architecture.
What's next for noongil
We are working with a B1 Parkinson's FB Group: https://www.facebook.com/groups/512119977204633 which has 14,000+ members (patients with Parkinson's and caregivers). We want to pilot noongil with a select group of members, iterate and improve the solution.
Next, we want to push Noongil a few directions.
First, we want to strengthen the causal engine so it produces more reliable and richer multi-day explanations for caregivers.
Second, we want to improve the caregiver product experience with a more polished dashboard, stronger care-plan workflows, and better communication tools.
Third, we want to harden the architecture for production by moving more model access behind backend-controlled pathways, improving deployment automation, and expanding clinician-facing reporting.
Longer term, we want Noongil to become a truly continuous care companion for neurodegenerative conditions, not just a daily check-in tool.
Log in or sign up for Devpost to join the conversation.