Aura

We all have 10,000 unread notifications and a smartwatch telling us we slept 64%. But who actually tells us what to do about it? Aura does

Comment

Inspiration

I’ve always been fascinated by how we can bridge the gap between machine and human wellness. I’m an AI automation nerd who loves Asian food and anime, and I realized that most health apps are boring. They just show you charts. I wanted to build Aura: a "Sentinel" that feels less like a spreadsheet and more like a coach who’s actually watching your back in real-time.

What it does

Aura isn’t just another health tracker that shows you a bunch of charts you’ll ignore. It’s an AI Wellness Sentinel. It uses the Gemini Live API to act as a real-time health coach. Instead of you checking your data, Aura checks it for you. It pulls your sleep, vitals, and calendar data in the background, and when you talk to it, it already knows if you’re stressed, sleep-deprived, or running late. It’s designed to bridge the gap between "dumb" data and actual human conversation.

How we built it

I powered the project with the Google GenAI SDK and the 2026 Agent Development Kit (ADK).

The Brain: I used a dual-model setup. Gemini 3 Flash handles the "Intel Team" (gathering data) because it's fast and smart. Gemini 2.5 Flash Live handles the voice interaction because it’s optimized for that sub-second, bidirectional audio flow.

The Architecture: I moved away from standard linear loops to a "Pre-Fetch" pattern. I use a ParallelAgent to harvest health metrics while the WebSocket connection is warming up, ensuring the AI is "briefed" before it even says hello.

The Memory: To keep Aura from having "Alzheimer's" every time I reboot the script, I integrated VertexAiSessionService and Google Cloud Storage to make user state and conversation history persistent.

Challenges we ran into

The biggest headache was the "1008 Policy Violation" error. I spent hours figuring out that the Live API is incredibly picky about which specific model IDs are used for bidirectional streaming. We also fought a lot with "Orchestration Tax". the lag that happens when multiple agents are talking to each other. In a live voice app, a 3-second delay feels like an eternity. I had to dive deep into async Python and MINIMAL thinking levels to shave off every millisecond of latency.

Accomplishments that we're proud of

I managed to get the response time down significantly, even while routing data from Tunisia to servers in the US. Getting a persistent session to work, where the AI remembers your name and your last heart rate reading after the program is closed, seemed like a huge win. I’m also proud of the architecture; it’s modular enough that I could easily plug in more "Intel Agents" (like nutrition or fitness trackers) without breaking the voice flow.

What we learned

It's my first time building with Google ADK, and it's great, better than prompting and low-code tools in customization, better syntax than LangChain, and with the Live API is a completely different beast than building a chatbot. I learned that latency is the killer of empathy. If the AI doesn't respond instantly, the "human" connection disappears. I also learned how to manage complex "State" in a multi-agent system, ensuring that data gathered by a background agent is actually accessible and useful to the "Voice" agent in real-time.

What's next for Aura

Aura is just getting started. My next step is to install in the hardware and integrate Unity to give Aura a visual presence—moving from just a voice to a 3D avatar that can react to your mood. I also want to expand the "Intel Team" to include local environmental data, such as air quality in Sfax.

Built With

Share this project:

Updates