Elderly face a tough challenge when it comes to managing their medications properly and being able to notify their doctor of their current medications. In the pharmaceutical world, drug labels and appearances change constantly, and the elderly are often unable to recognize their medicines. Combine this with the fact that many of the elderly have to take several pills, and it becomes clear that there is a real issue in medication mix-ups/misunderstandings of medications purposes. Finally, it is important that this information is correctly communicated to the doctor.

What it does

The app asks a few questions to the patient to characterize the pill. A database of prescription medicines is then searched and matched for medicines that fit the description of the patient's pill. The results are passed on to their primary care physician. Also, the assistant notifies the patient of the pill classification and allows the patient to find out more information. This allows the patient to remember the purpose of each medicine, and communicate to their doctor what medicines they have been taking.

How we built it

Elderly are often unfamiliar with current technology, such as taking pictures, performing complex searches, etc. Instead, we sought to make a hands-free method for pill classification. We used Google Dialogflow to create a google action that gets characteristics of the pill, and use the RxImage API to try and find a match in the Rx database. We also have a webpage that displays the results of the query to the physician, so that the physician can match the results with the most likely medication that the patient is describing.

Challenges we ran into

Our results are limited by the size of the database that has been curated. Also, the search involving imprints on the pills is very specific and can be improved. Finally, Dialogflow took some getting used to.

Accomplishments that we're proud of

The Google action is intuitive and easy to use. On proper descriptions, our accuracy on finding the matching pill is high, and the results are displayed in a nice visual format for the physician to view.

What we learned

We learned how to create a useful action for the google assistant, and all that is involved in modeling a proper conversation with the user.

What's next for Doctor Post

The next steps are to allow for a description of the classified pill through speech. At the moment, descriptions are obtained through a weblink, requiring the user/patient to have some technology knowledge. Also, more improved search techniques and a larger pill database can further improve the accuracy of classification. Finally, we would like to expand Doctor Post into other medical obstacles, such as drug interactions and being able to notify a health professional easily when a patient's condition changes.

Built With

Share this project: