Neighbor

dashboard
dementia

Inspiration

My grandmother lives alone, and I’ve always worried about her health, especially moments when she feels short of breath or anxious but doesn’t know whether it’s serious. Not everyone has a wearable device or constant medical supervision. We wanted to build something simple and accessible that could offer reassurance or signal concern using a device most people already have. That’s how Neighbor was born; it is a non-invasive way to turn everyday technology into an early awareness tool. Like a good neighbor, we're always beside you.

What it does

Protective Engagement: It acts as an automated "Safe-Gate," engaging users in conversation to prevent social isolation and shield them from fraudulent calls.
Passive Biometric Monitoring: While the user talks about their day, Neighbor’s backend analyzes 768-dimensional acoustic embeddings and respiratory energy envelopes.
Early Detection: By measuring Breathing Rate (BPM) and cross-referencing speech patterns against clinical dementia datasets, Neighbor generates a "Health Score" for families, signaling when it’s time to seek professional medical advice.

How we built it

Natural Language Processing: librosa, Wav2Vec2, Whisper, pyaudio, ollama
Backend: Python, PyTorch, Huggingface, Flask,
Frontend: React, Javascript
APIs: ElevenLabs

Challenges we ran into

PyTorch safetensors and "Future Warnings" with installation issues pertaining to required v2.6 but max available version downlable was v2.2
PyTorch compilation issues + NVIDIA CUDA issues.
Sending/recieving audio files via FastAPI (swapped for reliable Flask)
Preventing speech or coughing from being misclassified as breaths.
Designing a scoring system that is simple but meaningful.
How to approach live speech recognition while maintaining accuracy, speed, and API usage.
Connecting telephone calls to a server backend and relaying to frontend.
Privacy vs. Intelligence: Moving from cloud-based APIs to Local LLM Inference to ensure sensitive care data remains within the patient's home.
Normalizing dementia detection scores to prevent "false alarms" caused by model over-sensitivity in diverse acoustic environments.

Accomplishments that we're proud of

Successfully built a working engine that extracts respiratory patterns from raw audio without specialized hardware.
Implementing a legitimate Wav2Vec2-DementiaBank model into a consumer-grade application.
Designing a non-invasive health monitoring tool
Building an intuitive, detailed, and accessible interface
Viable dementia/no-dementia detection.

What we learned

We learned how subtle changes in breathing patterns can signal stress or potential respiratory issues. We also learned how challenging real-time audio processing can be and how important clarity is when building health-related tools.