Inspiration Everyone on our team already wears a fitness tracker. Your watch can tell you your resting heart rate but it has no idea that you skipped lunch, had four coffees, or spent the whole day without talking to anyone. The things that actually predict how long you live are mostly behavioral, and a wrist sensor is blind to almost all of them. A camera on your face isn't.
What it does Bryan runs on Ray-Ban Meta glasses and watches what you do, eat, whether you went outside, and how long you've been at a screen. When it sees something worth saying, it says it through the glasses' speakers in the moment, not in a weekly summary you'd ignore. At the end of a session it writes you a report with a score, a summary, and the photos of the moments it picked out.
How we built it The glasses stream camera frames to an iPhone over Meta's Device Access Toolkit, and the phone forwards them to a Mac over WebSocket. Every 1.5 seconds the Mac produces a "tick": a small object combining cheap pixel math we compute ourselves, like brightness and motion, with tags from Gemini Flash-Lite. A plain rule-based gate watches those ticks, and only when something persists does it escalate to GPT with the actual frames attached. Cheap observation runs constantly; the expensive model only wakes up when there's a reason.
Challenges we ran into Our first design called the vision model once a second with a 1 second budget, which sounded fine until we realized the model was not processing anything because of the latency. Separately, audio out the glasses kept failing because setting up the iOS audio session throws an error when a session already exists. We also had three people building in parallel and ended up with two independently written scoring engines that had to be reconciled.
Accomplishments that we're proud of The architecture which allows a model to run all day but respond rapidly. Two AI layers on two different clocks: Gemini Flash-Lite tags every single frame as it arrives, continuously and for almost nothing, while a plain rule-based gate with no model in it watches that stream for things that actually persist rather than flicker. Only then does it escalate to GPT, which gets the frames and the context and decides whether to say something out loud. Running a reasoning model on every frame would be unaffordable and too slow to talk to someone in the moment, and only running it on a button press means it isn't really watching. The split is what lets this run all day and still react in seconds.
Next Steps One of our team members Adam does health policy work and set up a meeting with health leaders at Emory to get their feedback, along with one of the leaders in the Rice Medical Consulting Club to work to bring it to those in the Texas Medical District.
Log in or sign up for Devpost to join the conversation.