Remedy

[Finalist] A hardware AI medication dispensing system that is simple, intuitive, and can prevent medication misuse.

Comment

Inspiration

Throughout the 21st century, millions upon millions of people, especially seniors, have lost their lives from medication misuse or missed medicine. According to a study conducted by John Hopkins University, in the United States alone, annually 250 000 people pass away due to medication misuse, missed medications and medical errors. After looking at some of the shocking statistics collected around the world regarding medication misuse, we decided to take action and create something that will prevent this issue altogether. That, is where the idea of Remedy originated

What it does

Remedy is a dispenser/medication manager that keeps track of your medication schedule and gives you your medication accordingly. Remedy uses artificial intelligence to recognize users by their faces. When the dispenser recognizes a user, and the user indeed has a medication planned at that time, the dispense system activates and dispenses the exact amount of medication needed. The dispenser features numerous different medication types that are swappable through the app, and an easy-to-use hardware and software system. A caretaker/helper can set the time, medication type and amount for another person. This person may be a young child, a senior, someone with Alzheimer's, a visually impaired person, or just someone who doesn’t want to risk missing medications. Once the user or caretaker has input the medication information, this then gets uploaded to the Remedy Dispenser Box. After spending 5 minutes adding medication types and uploading, your work is officially done, Remedy will handle the rest! At this point, the Remedy Box activates facial recognition and dispenses your medication according to your schedule. This entire process of recognition and dispensing happens in under 5 seconds. In fact, Remedy is so simple that the user doesn’t even have to touch the machine, all they need to do is stand in front of the camera! On top of this, Remedy also includes an AI voice system to accommodate people with visual impairment. Remedy also includes an LCD panel that displays prompts to accommodate people with hearing impairment.

How we built it

For the desktop application, we created a responsive graphical user interface using PyQt. This application is used when the user is inputting their medicine schedules. The application then saves the schedules onto a database. A serial port communication is utilized to transfer the user schedules to the raspberry pi. We implemented face recognition on the raspberry pi using OpenCV to make sure that the Remedy Dispenser can identify users properly. The raspberry pi also has a voice system where we used the google text to speech API along with the Google translate API for multiple language support.

On the hardware side, we utilized two Arduinos, one to drive the bright LCD and the other to drive the dispense and rotation servos. To control those Arduinos we also decided to use a raspberry pi 4 inside the box. On the raspberry pi, we feature a facial recognition AI system that allows the raspberry pi to determine the user, which allows the hardware to give them their medicine accordingly.

Challenges we ran into

We ran into many challenges for both the hardware and the software. For the hardware, the biggest issue we ran into was with the dispensing and spinning mechanism. During testing, a lot of the glue that we used got weakened, so the entire system lost its strength over time. We had to rebuild a lot of the parts again, including using multiple 3d printers to complete everything in time. Other than that, the LCD had a broken resistor, and it was in SMD format, so we had to get a new one and carefully solder it on again. This is our first time working with SMD components as well, so it took multiple tries but everything worked out in the end. On the software side, one of the biggest challenges was making the program compatible with both raspberry pi and windows. We had to make modifications to many parts of the code to get everything working. This was especially challenging since none of us had ever worked with a raspberry pi before. Other than that, implementing a proper face recognition tool and establishing/implementing a serial communication system protocol was also a challenge.

Accomplishments that we're proud of

We are very proud to have a fully functioning minimum viable product. We ended up learning many new technologies and overcoming challenges, but in the end we are proud to have accomplished our goal.

What we learned

Many of the libraries and frameworks we used for this project are tools we were using for the first time. As a result, we ended up learning quite a bit during this Hackathon. Some examples are how to utilize OpenCV for face recognition, how to create a viable dispensing system, how to use PyQt to create fully responsive desktop applications, how to create a database for custom data types/images, and how to create python programs for the raspberry pi ecosystem.

What's next for Remedy

We are very proud of Remedy, and after Hack the 6ix, we hope to expand this idea further until we end up with a polished product that can potentially help millions across the world. Although we had very limited time to complete this project, we had a whole list of ideas that we wanted to implement. One day, we hope to make all of these ideas a reality. Some of these ideas that came to mind were: *An advanced notification system through wearable technology, mobile applications and desktop applications (a vibration watch to notify when someone has medication, mobile and desktop notifications etc.)

  • Voice recognition system that allows the user to communicate with the machine
  • Android and IOS applications
  • Alarm feature to notify tampering of the machine
  • A feature that gives notifications (and potentially contacts an emergency line) when the person has not been taking their medications for an extended period of time
Share this project:

Updates