Inspiration
During our research on healthcare , we found that over 1 million die annually due to falsified and substandard medicines. These issues are much prevalent in poor regions where education is a privilege. To bridge this gap between fake medicines and poor regions we decided to build MediPill Ai.
What it does
Before the user consumes a pill, they can take a picture of it, upload it onto our web app, and input the name of the medication for which they were searching. Our trained OCR and Image Classification AI models will compare the user’s photo to existing data sets in order to match the medicine’s identity, and output a message stating whether or not it is recommended to take the medication.
How we built it
- The front-end of the web app was built using HTML, CSS, and JavaScript.
- We built our back-end in Python using the Flask framework, which made it easier to redirect and route URLS, and handle input from HTML forms. Moreover, we used Flask-SQLAlchemy to create a database that stores users’ information in an encrypted manner when they register for an account. We also used the OCR.Space API to create the AI model that extracts text from images, and Python’s pickle module for image recognition.
Challenges we ran into
- It was our first time using SQL to build a database. We wanted to ensure that users’ account information is stored safely and is encrypted, so we took a lot of time to learn and understand how to implement it.
- We had split up tasks for the backend between group members, so we wrote our code on independent files. Once we needed to merge the files together into one, we encountered errors.
- We faced challenges when connecting the front-end to the back-end, and so needed to make modifications to create a usable platform. -While setting up the OCR configuration, we had to play around with its different parameters a lot with ultimately achieving 89% accuracy for text recognition.
Accomplishments that we're proud of
-We were able to create a functional web app with accurate image recognition and OCR AI models, and incorporate many of the features that we had envisioned. -We successfully used computer vision in our application and built a fully functional mutli-modal classification system with over 85+% accuracy rate.
What we learned
- Creating a database in SQL and encrypt data for safer storage.
- Creating and training AI models. -We learned what OCR is and how it works. Understood why it fails due to blurry images , contrast and how to preprocess the images by adjusting grayscale , thresholding other to improve accuracy. Using cloud based OCR for better performance. -Learned how to use a pretrained CNN model to extract image embeddings.
What's next for MediPill AI
- Take user input related to their prescribed medication (brand, dosage, how often they have been advised by a healthcare professional to take it) to create a more personalized experience.
- Push reminders to notify the user about the medications that they need to take at a certain time.
- A chatbot where users can type messages in plain English and get appropriate responses.
Log in or sign up for Devpost to join the conversation.