DownLink

Helping elderly adults get their UpLink back.

Comment

Inspiration

My aunt falls multiple times a month, and these falls can be dangerous; if she fell without anyone knowing, it could take days until her personal support worker finds her and by then it would have been too late. Over 37.3 million falls that are severe enough to require medical attention occur each year according to the World Health Organization. Responding quickly to falls can help prevent fatalities, and that's where Downlink comes in.

What it does

Downlink is an all-in-one for detecting falls within the area that it's set up to cover. Downlink collects data sources including audio cues, and location information from wifi channel state information (csi) and two ultrasonic sensors which feed into a machine learning model to determine whether someone has fallen near by. When a fall is detected Downlink asks the user to confirm if they are ok, and if the user doesn't respond within 30 seconds or confirms they need help, then a warning sounds and their emergency contact is called. Downlink is private by design, having no cameras unlike traditional fall detectors and audio is only kept for at most 3 seconds before being deleted by a circular buffer, allowing for it to be placed anywhere without privacy concerns. Downlink both has a small footprint and doesn't require the user to wear anything, so it's out of mind until someone falls. While Downlink isn't functioning, it acts as another digital photo display letting Downlink easily blend in with your home. Also, Downlink is easily expandable (with ESP32s & ultrasonic sensors) and cheap to build, making it highly upgradable in the future.

How we built it

We broke our project down into modules, which collect data, predict & react.

The spatial module contains the ultrasonic sensors and wifi csi. Two ultrasonic sensors one horizontal and other one at a 30 degree angle pointing downwards captures distance from the sensor to the first object. Using both of them we can detect when something falls past the first one and then the second one within a certain timespan to determine if a fall may have occurred. This data was then combined with the wifi csi data obtained from the nexmon library and is fed into a machine learning model which can determine when a fall as occurred based on activity in it's surroundings.

The sound module, keeps a circular buffer containing the past 3 seconds of noise. When our custom-trained 86 million parameter audio spectrogram transformer (AST) machine learning model reports whether it thinks a fall event has occurred or there is a distress sound when called. Since this model is so large, we host it on vultr instead of the raspberry pi.

All of the above information is fed into a lightweight machine learning model on our raspberry pi which classifies each snapshot of data to determine whether a fall has happened. In the event of a fall, Gemini TTS is used to ask the user if they need help, and the display switches to a yes & no button. The user can reply by hitting a button on a display or verbally whether they need help or not.

Challenges we ran into

We ran into a few challenges while thinking about and building Downlink. We found out that wifi channel state information alone wasn't enough to detect someone's presence (if they are standing completely still then it can't see them), so added data from ultrasonic sensors to properly determine where people are. Later on, our fall detector wasn't detecting falls after training on 30+ sample sets of data that we created, so we continued to finetune our model and we found out that the phone acting as a microphone kept falling asleep removing that data source making our readings inaccurate. After we stopped the phone from falling asleep and added more data our model was much more accurate.

Accomplishments that we're proud of

After just reading about using wifi csi we were able to create a project that makes use of wifi to detect changes in its surroundings. For our first time working with hardware, we didn't encounter too many issues with wiring up the circuits we needed, and it was amazing how easily everything came together. Additionally, we are happy that get got a working final prototype, which detected falls as we envisioned.

What we learned

We got to dive into the research and methods to use wifi channel state information to detect changes within a defined area. Wifi bounces off surfaces from the router to the receiver, and when you move between them the signal takes milliseconds longer. These changes can be monitored and you can detect when something moves! I personally learned that Claude is really powerful, and it was able to do much more than we expected in a very short period of time, for example it was able to help us understand and start using the nexmon library quickly. Also we both learned that falling a lot really hurts, even if you're expecting it and do it on purpose.

What's next for Downlink

We're planning some upgrades to Downlink, given the chance to implement them. Adding additional nodes to the wifi csi network would improve Downlink greatly, and it could be done for a cheap price since any wifi emitting and receiving device (such as an ESP32) could work.

Built With

Share this project:

Updates

Submission history