RxConvert

Inspiration

We developed RxConvert after learning about non-literate communities being unable to understand their prescription labels and taking their medications at the wrong time and the wrong frequency.

What it does

RxConvert is a mobile app that uses optical character recognition to read scanned images of prescription labels. It uses these images to generate image-based instructions on how to take a prescription. For example, a prescription label containing the phrase "by mouth" will become a set of two images, the first showing someone holding a pill and the second showing that person putting the medicine in their mouth.

How we built it

We built RxConvert using the Flutter SDK and Google ML Kit's text recognition API.

Challenges we ran into

Since prescription bottles are curved significantly, we found it hard to scan without cutting parts of the label off in the image. We first made headway on this issue by taking three pictures of the bottle (left, middle, right) and looking for prescription label keywords in each image. However, some prescription labels contain keywords cut off by line breaks, an issue we are currently solving. Furthermore, Google ML Kit's text recognition API was not 100% accurate, so we had to implement fuzzy contains checks for token recognition.

What's next for RxConvert

We are currently adding support for more languages, such as Burmese. As mentioned, we are also solving token recognition issues caused by line breaks and continue to improve our bottle curvature-induced text recognition issues.

Built With

• dart
• flutter
• google-ml-kit
• ocr