-
When users submit the picture of the pill to the website they get the confidence score , the name of the pill and can ask qns with the AI
Inspiration
Our idea was inspired by the ubiqitous problem that we face in many households , mislabelling our pills. Most prescription for the pills which include what the pill is for and the freqeuncy & number of pill to take per day is often printed on a plastic envelope in very small printing. Many households , over time , misplace the pills into different plastic envelopes or even lose their plastic envelopes.
This problem is even more significant in elderly households where the elderly living alone misplace their plastic envelopes due to ageing forgetfullness , the labelling on the plastic envelopes may fade with time or the elderly may not even be able to read the printing on the pills. This gives rise to severe complications such as mistakenly taking the wrong pills or the wrong quantity of pills which may lead to medication overdose .
Another reason why we chose the elderly as our main target audience is the fact that they usually take mulitple pills due to a numerous number of medical conditions that arise due to ageing such as type 2 diabetes , hypertension and high cholesterol. This escalates the risk of the elderly taking the wrong pill coupled with other factors such as poor eyesight and increasing forgetfullness.
What it does
The elderly would have to take a picture of pill and upload it onto our webiste which through Machine Learning they are able to identify the pill that they are supposed to take and the freqeuncy of which to take it . They are also able to chat with the integrated AI to know more about the pill and clarify any doubts they may have of the pill.
We used online datasets to create and train our machine learning model to detect pills from images with an accuracy of above 90% . We have integrated Google Gemini into our webiste to help clarify any questions the elderly may have of the medication as well to improve the functionality of our website.
Furthermore, we have tried to simplify the web application and avoid as much clutter as possible to have a smoother user experience for the elderly to navigate the website and promote the overall effecient usability of the website. The name of our website is PillVision and the name of our integrated AI is Pill.AI
How we built it
We used Tensorflow to build the CNN model (Convulational Neural Network) and StreamLit to create the frontend website to improve the accessibilty of the website. Furthermore , we incorporated Google Gemini onto the website elucidate questions the elderly may have.
Challenges we ran into
It is our first time using Streamlit to create a frontend and it took us long hours to create a clutter-free and intuitive website that the elderly would be able to use.
Accomplishments that we're proud of
We are proud that we were able to train our Machine Learning Model to an accuracy of 90% within a short period of time and also integrate Gemini into our website as well.
We are also immensly pleased for being able to create a simplistic yet highly beneficial tool to improve the lives of many elderly in our ageing society.
What we learned
We have learnt how to use StreamLit and how to more effectively train a machine learning model within a shorter period of time .
What's next for PillVision
In the future , we aim to transform into an app which would be more convenient and accessible for users .Furthermore , we aspire to have a larger dataset and include more pills of different varieties as well. Lastly, We would also like to integrate a multilingual interface into our website/app to help reach out the elderly who may not be proficient in English in our society and expand our project to create a global impact to countries with lower English literacy rates.
Log in or sign up for Devpost to join the conversation.