Currently, patients in hospitals notify staff of their caregiving needs by making a call using a phone, or by pressing a button that requests the presence of a nurse. We wanted to improve on two aspects of this system. First, injured or elderly patients may not be physically able to reach a phone or a button. Second, valuable hospital time is wasted when the nurse must visit the patient for every request, find out what the request is, and then leave. Our system allows patients to send requests with a minimum of physical movement, and for caregivers to know what their request is instantly through the visual request dashboard.

What it does

Once a Leap Motion controller with Gesture Based Caregiving is mounted by a hospital bed, bedridden or physically less-able patients can simply make gestures in the air to navigate a caregiving request menu. The request is made and prioritized based on severity (request for water, low priority - request for painkillers, high priority), and is fed to a web-based visual dashboard to allow the caretaker to manage requests from multiple patients.

How we built it

The gesture-recognition is done using a Leap Motion controller and the Leap Motion Javascript library. The visual dashboard is built in Javascript using D3.js. For high priority requests (such as health emergencies), a notification is sent to caregivers over text using the Twilio API.

Challenges we ran into

While our original plan was to store requests using JSON files and read then using D3.js, it proved to be difficult to get working. Instead, we used HTML5 local storage to record values from the gesture recognition page and transfer them to the requests visualization page.

What we learned

All about using Leap Motion and Data Based Documents in Javascript!

What's next for Gesture Based Caregiving

We would like to refine the user interface, add detailed navigation menus, and add voice command control with Amazon Echo to further ease the process of requesting care.

Share this project: