After long research about our healthcare system in our country, Singapore. We were utterly were shocked by the devasting effects of merely not being able to read a text on a piece of paper, we were utterly shocked how people were neglecting their health and detering away from seeking treatments because of a complex healthcare system and we were shocked that people in our country just did not know where to go to seek help. All these caused a snowball effect of impacts, from small things such as headaches and more frequent falls to even death. These devasting impacts derived our motivation and passion in wanting to make a change, in wanting to use the skills our society has equipped us, the skills our teachers have taught in our classroom to give back to our society helping people like ourselves, our teachers, our family and even the elderly.

What it does

The user can upload an image of their prescription label, the text which will then be detected by our programme and cross-reference back to our database and extract out the drug name and side effect. This information will then be output in an enlarged format and in a translated language chosen, it will also output voice recording in case the individual cannot read. Chatbot allows users to ask any questions regarding the covid situation in Singapore from the number of cases to vaccination information. The advantages of this platform are that it is convenient and all the relevant information will be there for the users to view. Additionally, it is very versatile which allows us to be able to personalized the user's experience easier. We have incorporated dialogflow which translates human language to robot language for the chatbot to understand and provide relevant replies for the user. We have gathered FAQ pages from different websites into our chatbot so that common ask questions will be registered and the relevant answers will be provided for the users. We have also used AI Janis which will inform us about the messages and can bring a human into the loop. Hospital search was built with HTML and CSS. Hospital Search is convenient as it is convenient with all the hospitals information being able to be viewed individually, Additionally, with it's a simple interface, it allows users to be able to use it instantly. Last but not least, there is a hospital's speciality that allows users to know the speciality of each hospital which can be of big advantage to users in understanding the hospital even better.

How we built it

Our prescription tool consists of two programs. The first program utilizes EasyOCR, TesseractOCR, GoogleTrans API, Google Cloud Text-To-Speech API. Our second program is to validate the input, to see if the input is actually a prescription label. We train our second program with CIFAR10, a dataset with common objects seen like cars. Using further techniques such as data augmentation and transfer learning using VGG19, we manage to do just it. The thinking behind this can be seen in our pitch video and google collab link. Our chatbot was built using a natural processing language utilizing Dialogflow and, we further added FAQs from 10 different websites using the google sheets possibility in Dialogflow. Lastly, our hospital search is mainly relying on government and hospital websites along with HTML and CSS techniques and skills.

Challenges we ran into

We were new to the idea of incorporating Artificial Intelligence into our idea because we believed that AI can make the most impact and is most beneficial to the issue we have chosen. Due to the lack of knowledge and skills we spent the sleepless night researching about AI, the number of articles on RESNET, DENSENET, neural networks, datasets definitely drove some thoughts on giving up trying to do something we do not know how to do. Continuing on to the competition period, we were faced with many challenges and spend hours troubleshooting them. The idea of totally wiping this idea and forgoing the idea of doing something we thought we were incapable of doing definitely came about again. We pushed ourselves and encourage each other to persevere and never give up. Fortunately, we have solved most of the problems and have successfully provided a result.

Accomplishments that we're proud of

We are proud of the result we have made. Having to create a full usable website in just these days with many features from the prescription aid to the hospital search. Being able to bring our plan to life just makes it even better. Additionally, being to learn so many different things in just these days made us understand that we should not underestimate ourselves and set higher goals for ourselves.

What we learned

Other than learning AI and how to persevere, we learnt how to put together a project. In our school, both of us were taught skills on how to code basic. However, we were never exposed to putting the skills together to form something, during this competition we learnt the process of brainstorming an idea all the way to the final output. Seeing how all these different skills from different lessons were put together is just an amazing sight, but not an easy journey.

What's next for Healet

It will definitely be getting a custom domain or using websites like Heroku to shift our flask web app to a tangible web app that the public can assess.

Built With

  • bootstrap
  • cirfar10
  • css
  • dataaugmentation
  • dialogflow
  • easyocr
  • googlecloudtexttospeechapi
  • googletransapi
  • html
  • python
  • tessseractocr
  • transferlearning
  • vgg19
Share this project: