HeartBridge

1. About the project

Inspiration

Parenting a neurodivergent child (e.g., with Autism/ASD) is a journey filled with love but often accompanied by isolation, uncertainty, and "after-hours" crises where professional help isn't available. We wanted to build a bridge—HeartBridge—that connects the daily, granular struggles of parents with immediate, clinical-grade behavioral insights. Inspired by the need for "predictive" rather than just "reactive" care, we envisioned an app that doesn't just log behaviors but understands them, acting as a divine support companion for children like "Sammy."

What it does

HeartBridge is a comprehensive AI-powered behavioral health platform that provides:

Predictive Support Dashboard: analyzing daily metrics (Sleep, Meltdown, Communication) to forecast behavioral trends and offer proactive advice.
Multimodal Behavior Analysis: Parents can upload video/audio of a behavioral event. The app uses Gemini 3 Flash to analyze the visual and audio cues, breaking down emotions (Happy, Angry, Nervous, etc.) and generating an immediate, clinically-grounded "What To Do" guide.
AI Expert Consultation: Simulates conversations with specialists (BCBAs, OTs) using Gemini 3, allowing parents to "chat" with specific expert personas about their child's specific history.
Generative Therapeutic Media: Uses Imagen 3 (Gemini 2.5 Flash Image) and Veo to generate calming, neuro-affirming visualizations and videos on demand for the child or parent.
Community & Resources: A curated feed of therapies and tips, backed by an intelligent chatbot assistant ("Nomi") that uses the child's name and specific context.

How we built it

We built HeartBridge as a React Native-style PWA using TypeScriptand Tailwind CSS for a soft, accessible "neuro-calm" UI. The core intelligence is powered by the Google GenAI SDK:

Behavioral Reasoning (Gemini 3 Flash Preview): We utilized the gemini-3-flash-preview model for the heavy lifting—analyzing complex user logs and chat interactions. Its high reasoning capability allows it to act as different "Expert Personas" (e.g., a BCBA vs. a Speech Pathologist) by maintaining distinct system instructions and context windows.
Video & Image Analysis: The analyzeMedia feature sends base64 encoded video frames/images to Gemini, requesting structured JSON output (using responseSchema) to categorize emotional states and generate actionable steps.
Generative Media: We integrated gemini-2.5-flash-image to generate supportive community illustrations and Veo (veo-3.1-fast-generate-preview) to create therapeutic background videos, enhancing the app's sensory-friendly atmosphere.
Structured Data: We heavily relied on Gemini's responseMimeType: "application/json" to ensure that AI insights could be parsed directly into UI components like charts and progress bars.

Challenges we ran into

Prompt Engineering for JSON: Getting the model to consistently output valid JSON for the "Behavior Report" (with specific percentages summing to 100%) required careful tuning of the responseSchema and system instructions.
Handling Multimodal Inputs: converting user-uploaded video files into a format (Base64/Frames) that the Gemini API could process efficiently within the browser client.
Balancing Tone: ensuring the AI sounded clinical yet deeply empathetic ("close to the heart") required several iterations of system prompt refinement.

Accomplishments that we're proud of

Successfully integrating Veo for video generation, which is a cutting-edge feature.
Creating a seamless "Consultation" flow where a parent can upload a video of a meltdown and instantly get a breakdown and a chat session with an AI specialist who "watched" the video.
The UI design—achieving a "Glassmorphism" look that feels premium and calming, essential for our target audience.

What we learned

We learned that Gemini 3 is exceptionally good at "Role-Playing" complex clinical personas when given robust context. We also discovered the importance of multimodal context—text descriptions of behavior are often insufficient; seeing the behavior (via video analysis) changes the quality of the advice drastically.

What's next for HeartBridge

Real-time Audio Monitoring: Utilizing the Gemini Live API to listen for auditory triggers (crying, yelling) in the background and offer real-time de-escalation voice coaching to the parent.
Wearable Integration: Connecting to real watch data for HRV (Heart Rate Variability) to improve the "Predictive" algorithm.
IEP Generation: Using the gathered data to auto-generate Individualized Education Programs (IEP) for schools.