lisa

Inspiration

Prompt 2- MS (Healthcare) The strain on caregivers for MS patients and others that are made dependant by debilitating sickness is immense. Lisa aims to be an assistant that can watch when you are not around and identify you when something is wrong based on a deep learning model.

What it does

Uses computer vision and speech analysis to identify when the patient needs assistance or is at risk then notifies the designated caregivers that their attention is required.

How I built it

We built python scripts that use Tensorflow and OpenCV for computer vision intended to run on a raspberry-pi. We built a demo of the application using figma prototyping.

Challenges I ran into

Using the YOLOv3 (You Only Look Once) algorithm for image analysis was a difficult algorithm to wrap our head around in 24 hours, but eventually we understood it's functionality. OpenCV documentation was sparse or difficult to to find and made the process quite difficult. We had a lot of difficulty connecting the raspberry-pi and eventually were unable to use it.

Accomplishments that I'm proud of

Using deep learning models for analysis Using twilio to send texts as alerts Using figma prototype mode for an interactive experience

What I learned

The constraints of a pre-trained model The true expense of training a large model (time and computational)

What's next for lisa

-Developing mobile app -Finishing raspberry-pi based hardware

Built With

• figma
• numpy
• opencv
• python
• raspberry-pi
• tensorflow
• twilio