PulseAI

Collecting data with Presage
UI
UI with patient in camera
Emotion detector!

Inspiration

After completing their treatments or checkups at the hospital, many patients travel to be in the comfort of their own home. They rely on family to regularly check up on them, but their families cannot stay at the patient’s bedside 24/7 because of work, school or other essential tasks. That means, at any given time, there are many patients who are unattended to, which is obviously not ideal. We wanted a program that could come as close as possible to mimic a 24/7, 1-on-1 caretaker-patient ratio so that everyone is accounted for should issues arise.

What it does

PulseAI is an AI-powered home monitoring system that watches elderly patients 24/7 to detect medical emergencies before it's too late. Using real-time vital signs monitoring, computer vision, and voice analysis, PulseAI detects strokes, and health decline - then automatically alerts family or calls 911 within seconds. Our conversational AI companion conducts daily wellness check-ins while Gemini generates comprehensive health reports. When grandma feel uneasy, PulseAI ensures help arrives in seconds, not hours.

Technical Information

We built a multi-server architecture: a C++ server integrating Presage SDK for real-time vitals (heart rate, breathing, stress, movement), a Python Flask backend for AI analysis and alerts, and a React.js frontend. We bridged C++ and Python using TCP sockets, and used WebSockets for real-time frontend updates. Google Gemini powers multi-modal AI analysis, ElevenLabs provides our AI companion voice, and we integrated a phone API to automatically dial 911 during critical alerts.

Challenges we ran into

Our biggest challenge was integrating Presage SDK (C++ only) with our Python backend. We solved this by building a C++ wrapper with TCP socket communication, running on WSL. Optimizing ElevenLabs for real-time conversational AI without delay required significant work. Integrating automated 911 calling was complex - ensuring reliable emergency calls with correct patient information while preventing false positives took careful calibration.

Accomplishments that we're proud of

We successfully integrated Presage SDK despite limited language support by building a custom TCP socket bridge. Getting automated 911 calls working reliably could genuinely save lives. Our conversational AI feels warm and caring, not robotic. Most importantly, we built a system addressing a deeply personal problem - family members who couldn't be helped in time, leading to devastating regrets. This project is personal, not just a hackathon submission.

What we learned

We learned inter-language communication using TCP sockets and real-time data streaming with WebSockets. Building a C++ wrapper integrated with Python taught us valuable system architecture lessons. We gained experience with multi-modal AI (Gemini analyzing video, audio, and text simultaneously) and real-time voice synthesis. We discovered how AI tools can accelerate full-stack development. Most importantly, we learned that with determination, we can build life-saving systems in 36 hours.

What's next for PulseAI

We're expanding monitoring to include stress levels, blood oxygen (SpO2), and temperature. We'll add automatic incident report generation with video clips, vitals, and AI analysis for paramedics. Multi-user support will let families monitor multiple relatives. We're building automatic doctor scheduling for non-critical concerns and medication tracking with overdose detection. Future integrations include smart home devices (auto-unlocking doors for paramedics) and wearables for 24/7 monitoring beyond home cameras. Our goal: prevent the 125,000 yearly deaths from unmonitored elderly emergencies.