KinSight: Elder Behavioral Safety Intelligence

An ambient AI system that moves beyond visual pill detection to passively monitor and understand elder behavior, preventing cognitive-driven medication errors (like repeated dosing) in real-time.

Comment

About the project

💡 What inspired us Caring for elderly loved ones, especially those experiencing memory decline, is a constant source of anxiety. A routine as simple as taking medication can become dangerous if an elder forgets and accidentally double-doses. As IT students from HCMUS, we wanted to build a solution that doesn't just "watch," but actively "protects." KinSight was born to bridge the gap between elderly independence and caregiver peace of mind.

⚙️ How we built it (The Core MVP Logic) KinSight is a real-time behavioral safety AI system consisting of two main interfaces: the Elder Kiosk (a pill-scanning screen) and the Caregiver Dashboard (a remote monitoring hub).

Instead of complex data pipelines, our MVP focuses strictly on a Cognitive Safety Loop. When an elder holds their medication in front of the camera, our AI (powered by the Groq API) instantly recognizes and verifies it.

We modeled our emergency trigger using a clean, threshold-based logic. Let $W$ represent a cognitive warning event (e.g., attempting to take the same pill twice). The system state $S$ automatically transitions to an Emergency state ($E$), triggering an instant Agora WebRTC video call, if the accumulated warnings reach our safety threshold $\tau$:

$$S = \begin{cases} E, & \text{if } \sum W \ge \tau \ N, & \text{otherwise} \end{cases}$$

For this MVP, we set $\tau = 2$. This allows the system to provide a gentle audio reminder on the first mistake, but instantly react—sounding an alarm and connecting a live video feed—if the dangerous behavior repeats.

To bring this to life, we utilized Next.js and Tailwind CSS for a responsive frontend, deeply integrated with a lightning-fast FastAPI (Python) backend.

🚧 Challenges we ran into Taking KinSight from localhost to the cloud was a real battle:

  • Dependency Hell: Deploying our backend to Render caused severe Python version conflicts. We had to strategically lock libraries like httpx and downgrade environments to ensure our AI SDK compiled and ran smoothly.
  • Next.js Prerendering Traps: Vercel repeatedly failed our builds due to dynamic routing hooks (useSearchParams). Figuring out how to properly wrap our components in <Suspense> boundaries to protect the render tree was a stressful but incredibly rewarding fix.
  • Mobile UI/UX: Preventing the camera interface from breaking or getting hidden behind mobile browser toolbars required meticulous viewport and CSS adjustments.

🎓 What we learned Our biggest takeaway is that "working on my machine" is only half the journey. We gained invaluable, hands-on experience structuring a production Monorepo, securing environment variables, and weaving three distinct platforms (Next.js, Groq AI, Agora WebRTC) into one unified cloud product. Seeing our project officially go "Live" on the internet was an unforgettable milestone for our team!

Built With

Share this project:

Updates