We believe that electronic health records are a technological necessity going into the next decade. The digitization of records allows for rapid access to patient health history, increased privacy and security of information, and even predictive healthcare utilizing machine learning. Many hospitals in developing and rural areas haven't transitioned from handwritten records to electronic health records yet. Through developing mediScan, we hope that we can help others digitize their medical information towards a healthier and smarter future.
What it does
Our product, mediScan, allows the user to take a picture of any handwritten document through an android app. The user will then highlight areas of handwritten text information to then convert them into easily-readable spreadsheets (.json files) through Google Cloud Vision.
How we built it
The android app was built using Android studio; we implemented a scan button to launch the camera app. Once the camera app is launched, the user can take a picture. Afterwards, we create an intent to go to another activity where the user can construct a rectangle around any area that is within the boundaries of the imageView (which displays the photo). After drawing a legitimate rectangle, we prompt the user to enter a String that corresponds to the handwritten data. Once we have the fields, rectangle coordinates, and image (base 64), we put this into a json which can be sent to the cloud instance. The response consists of a JSON document with the digitized form. This cloud instance was processed with Google Cloud Vision API in python, and a JSON document with all of the results was sent back to the android app for the user to receive.
Challenges we ran into
Creating the rectangle selector for pinpointing different Placing the python script to call the API on the cloud was difficult as the documentation and setup were unclear. It took us a while to configure the cloud properly for use. In addition, the script took a while to be registered on the cloud which slowed down our debugging process. Also, the Google Cloud Vision process had common errors at times, which had to be manually fixed through string parsing algorithms.
Accomplishments that we're proud of
We built a clean, functional android app and implemented a Google Cloud API with it.
What we learned
Coding is hard. Staying awake is harder.
What's next for mediScan
Implementation of natural language processing to improve Google Cloud Vision accuracy.