Medicine Madness Management

Simply SNAP, SAVE & STAY CALM - MMM will read, understand and remember your medication schedule for you

Comment

PROJECT WRITEUP

Inspiration

1 billion or 15% of the world's population have a (diagnosed) disability. 1 in 4 adults in the United States have multiple chronic illnesses which are often disabling. For individuals over age 65, that number is tripled to 3 quarters of seniors. To add to the overwhelmingness of managing their conditions, the persons with disability or their caregivers often have to take charge of a complicated medication regimen. Each medication comes with its own dosage and intake requirements, as well as multiple warnings or instructions. Furthermore, for persons with disabilities and chronic illnesses, dosages and medications to take can be constantly changing over time or based on the person's current condition. When patients don't stick to their prescribed medicine regimen, they are quickly classified as non compliant and often dismissed. However, a closer look found that 80% of these individuals have 3 or more chronic illnesses and 70% have mental illnesses. 70% also realise the need to self-manage their medication to treat or manage their conditions but are unable to do so without assistance.

What it does

Thus, our Medicine Madness Management APP serves to streamline the user's medicine management process. Enabling the user to easily manage their own medicine regimen and increasing their independence as they require less support from their caregiver in this regard. Alternatively, the APP could also be utilised by a caregiver of a person with disability to reduce their already heavy caregiving burden.

The user can simply use the APP to take or upload a photo of their prescription or doctor's instructions. The APP will then use an AI model to pick out key information such as the medication name, when or how often the medicine should be taken, dosage, and any relevant warnings or instructions such as "do not take on an empty stomach". The user or their care provider has the option to check through and ensure accuracy if they wish. Then, reminders will be scheduled based on the gathered information to alert the user to when they should take each medication along with the appropriate warnings or instructions.

How we built it

The source code was built on python using a character image recognition software and facebook’s fasttext to identify the different sections of prescriptions and medication labels and interpret it. The use of fasttext enables the medicine names to be extracted specifically in scenarios where the image recognition software feedback garbled characters. This design gives it an edge over general purpose image recognition software. A wireframe of the APP's user interface was also made to provide a demonstration for the eventual look of what the APP will do.

Challenges we ran into

Due to the short 24h time frame of the project, there was insufficient time to train a comprehensive machine learning model and connect the front-end UI to back-end software to make a full-fledged mobile application. Additionally, there were limited images of prescriptions and medicine labels for obvious privacy reasons. Thus, we had to scramble around our medicine cabinets to take photos to use as training examples for the model.

Accomplishments that we're proud of

We are proud of the algorithm we have learnt how to and managed to build in 24h. It is trained to read an image and detect, not just, English words, but also medication medical names and specific instruction (compared to general character image recognition software). Additionally, it is unique amongst other image word detectors in that it accesses a medical database to be able to detect the medical names also - essential for our application.

What we learned

We learnt about NLP, image recognition and Web API all in 24 hours, stitching together the different components together and creating algorithm rules to generalise our product to many medicine tags. As engineering students, it was also one of our first times doing APP UI wireframing and business case proposals which we are glad to have had the opportunity to learn to do.

What's next for MMM

Immediate goal: Tying the front-end to the back-end to produce a full useable mobile application Secondary goals: Adding in-APP features such as medication history, health and well-being, symptoms check-in etc. For a more comprehensive management APP which helps users stay on top of all aspects of their medication and condition. Extensions: Target other industries beyond medicine (such as groceries, to help people with dyslexia, or other visual disabilities to shop through a image-to-text-to-audio process)

Built With

Share this project:

Updates