Inspiration
A long, healthy life isn’t built in big decisions. It’s built in small daily moments: the energy drink at 11 PM, the reels in bed, the rushed rep at the gym. Those moments add up. Most health apps only work if you log everything, and most people stop logging. Personal coaching works, but trainers and longevity programs cost more than most people can pay.
I wanted a coach that is simply there in the moment. Cue runs on smart glasses. It sees what you see, speaks up before the bad choice instead of after, and needs nothing typed in. It logs for you: late caffeine, screen time before bed, and every rep with its form. But glasses can’t see how your own body moves, so I built a clip-on sensor that tracks your movement and coaches workouts rep by rep.
No manual logging. No expensive coach. Just the right cue at the right moment.
What it does
Cue has three parts that all feed one Longevity Score.
- Gym spotter
Cue Clip: counts curls, times each rep (“Slow down on the way down”), catches half reps (“All the way up and down”) and buzzes when form slips. Glasses camera: counts squats. No hardware? The web app counts squats and curls from a laptop camera and scores each rep out of 100.
2.Longevity coach The glasses log good habits straight from what they see: water, morning sunlight, fruit and veg, brushing, sunscreen, vitamins, stairs and time outside.
- Habit breaker Cue catches late caffeine, doomscrolling, screens at night, vaping and nicotine pouches, sugary drinks, betting apps, late meals and evening alcohol. Then it speaks a short swap in your ear.
Advice depends on the time of day. Coffee at 8 AM gets logged; coffee at 11 PM gets “Put it back.” For nicotine, Cue coaches you through the craving and offers the free quitline, 1-800-QUIT-NOW.
Longevity Score and companion app Everything rolls into one score with plain reasons. The app shows a Today checklist from morning to night, a Break column for bad habits, streaks and a weekly view. It updates live from the glasses and clip. A demo clock lets you jump through the day to see how the advice changes.
Cue is wellness coaching, not medical advice.
How we built it
I built Cue solo in 36 hours, with Claude as an AI coding assistant. The pipeline runs in six steps:
See: The Meta glasses stream first-person video over a WhatsApp video call to a Windows laptop, where the Cue app screen-captures the call window. Understand: Every 3 to 6 seconds, one frame goes to a vision model through an OpenAI-compatible API. The live demo uses Google Gemini Flash-Lite. I also served the open-source Qwen2.5-VL-7B on AMD Radeon Cloud with ROCm and vLLM; because the API is the same, the code runs on either model. One prompt asks which of about 20 habits are happening, and whether you are outdoors, in daylight or in the dark. Decide: A rules engine turns that answer into logs and nudges. It uses the time of day, cooldowns, and two-frame confirmation for things like vaping, so one bad frame never triggers a warning. Coach: Windows text-to-speech plays into a virtual audio cable that WhatsApp uses as its microphone, so Cue speaks through the glasses’ open-ear speakers. Feel: The Cue Clip is an Arduino Uno with a Grove accelerometer and a servo that buzzes for haptic alerts, in a 3D-printed case. It streams tilt 50 times a second over USB. My rep counter learns the resting position, so it counts curls however the clip is mounted, and it times both halves of every rep. Track: The laptop app keeps the day’s score and serves it locally; the companion web app polls it every 2 seconds. The web app also runs MediaPipe Pose in the browser for camera rep counting, all on the device.
Quality: 27 automated tests, including a simulation of 20 random clip mountings that must each count exactly 6 curls.
Challenges we ran into
Getting video off the glasses. I didn’t have direct camera access, so I streamed through a WhatsApp video call and screen-captured the call window. Getting a voice back in. It was the same problem in reverse. I routed text-to-speech through a virtual audio cable that WhatsApp uses as its microphone. False alarms. One blurry frame could make Cue think you were vaping. Two-frame confirmation and cooldowns fixed it. The clip never sits the same way twice. Each mounting gives a different angle, so the rep counter learns the resting position before it counts. Doing it all solo in 36 hours: hardware, vision, voice and the app at the same time.
Accomplishments that we're proud of
Real hardware, built solo this weekend. The Cue Clip counts real curls and buzzes in your hand. In a live set, it counted 10 of 10 curls and flagged the rushed ones. A rep counter that works however you clip it on. It learns the resting position, and it passed a simulation of 20 random mountings, counting exactly 6 curls every time. It’s backed by 27 automated tests. Coaching that knows the clock. The same coffee gets logged at 8 AM and gets “Put it back” at 11 PM. No warnings from one bad frame. Serious nudges, like vaping, need two frames in a row before Cue speaks up. A real voice in your ear. I routed text-to-speech through a virtual audio cable into the WhatsApp call, so Cue speaks through the glasses’ own speakers. An app that fills itself in. The companion app updates live from the glasses and clip, with nothing typed. Runs on open-source models too. The same pipeline works with Qwen2.5-VL-7B, which I served myself on AMD Radeon Cloud. A demo anyone can try. Put on the glasses, pick up a drink, do a curl. No glasses? The web app counts reps from a laptop camera.
What we learned
Simple beats fancy for reps. A clean tilt angle, good thresholds and learning the resting position counted reps better than anything complex. Coaching is about timing. The same advice is useful at 11 PM and just noise at 8 AM. Vision models see well, but not perfectly. They name objects well in a messy first-person view, but every warning needs a second look to avoid false alarms. Design for swapping models. Using an OpenAI-compatible API let me move between Gemini and my self-hosted Qwen without rewriting code. Honest labels build trust. The app marks what is live and what is sample data.
What's next for Cue
Cue Band: Turn the clip into a wireless wristband: an ESP32 with Bluetooth and a battery. The Bluetooth code is already in the project. A band stays on all day, so Cue can buzz you anywhere, not just in the gym. Direct camera access: Use the glasses’ camera directly instead of a video call, for all-day use. Privacy: Cue never saves frames. Today, each frame is sent to Google’s Gemini API to be read. Next, the open-source model runs on hardware we control, so frames never leave it. More of the day: Add sleep and steps from the Cue Band or wearables people already own, protein from meal photos, and coaching that adapts to each person’s goals.
Log in or sign up for Devpost to join the conversation.