Maxwell, Interactive Health Assistant

Inspiration

I spend way too many hours staring at screens without noticing how tired or tense I actually get. Most health monitors require smartwatches or extra gadgets, so I wanted to see if an ordinary laptop webcam could spot fatigue, rising stress, and vital signs entirely on its own. I also got inspired by Baymax from the Disney movie "Big Hero 6" and his role as a robotic personal health companion and tried to mimic many of his features in the movie.

What it does

Maxwell turns a basic webcam into a quick health monitor by checking eye blinks, squinting, and brow tension to estimate stress and eye strain. It catches yawns and slouching in real time while estimating your heart rate just by tracking subtle color changes in the skin on your forehead. You can type or speak symptoms directly into the screen, and Maxwell combines your live vitals with Gemini to give clear triage advice and recovery tips. It also includes a guided box breathing circle to help you reset when your stress spikes.

How we built it

I wrote the entire project in Python without heavy web frameworks, using MediaPipe and Face Landmarker to track facial points in real time. I used NumPy and SciPy to run bandpass filtering and an FFT to pull heart rate frequencies from the forehead camera feed. I connected the Gemini API through SerpApi in background threads so queries never freeze the live video. I also added speech recognition and text to speech so you can talk or type directly on the screen, and I rendered an interface and diagnostic HUD as much as I could inside OpenCV.

Challenges we ran into

Not all the features work as I intend to or am limited by simply using visual evaluation for health such as heart rate estimation. I'm currently unsure how to overcome this technological obstacle simply by running on my local computer and using computer vision libraries.

Accomplishments that we're proud of

I am proud of getting an optical heart rate estimate to work on an ordinary laptop camera with zero extra hardware. I was able to keep the live video, face tracking, and on screen typing smooth at thirty frames per second. Building an interactive health tool that feels responsive and genuinely useful right on my machine feels like a project with enormous potential that given more time to work and develop could become widespread to become part of like a physician checkup.

What we learned

I learned a lot about how computer vision libraries and brainstorming what a health AI companion would interact, features, and work like.

What's next for Maxwell, Interactive Health Assistant

It would be cool if by using infrared vision we could gauge a person's temperature and be able to also diagnose whether they have conditions like a fever.

Built With

• artificial-intelligence
• audio-processing
• biometrics
• computer-vision
• digital-signal-processing
• face-tracking
• fast-fourier-transform
• gemini-api
• healthtech
• human-computer-interaction
• machine-learning
• mediapipe
• numpy
• opencv
• python
• pyttsx3
• real-time-processing
• rppg
• scipy
• serpapi
• signal-processing
• speech-recognition
• threading
• triage