VisionGuide

• 

  Main dashboard

• 

  Detection of expiry dates

Inspiration

Small expiry dates and labels on medicine strips and food packaging can be difficult to read for people with low vision. A wrong reading can be dangerous, so an accessibility tool should not only extract text but also communicate when an image or reading is unreliable.

What it does

VisionGuide is a camera-assisted web prototype for reading text on medicine and food packaging. Users can select a category, point their camera at an item, or upload a photo. The app provides image-quality guidance, processes the image, extracts text using OCR, and attempts to identify expiry-date information. When the reading is uncertain, it flags the uncertainty rather than presenting the result as confirmed.

How we built it

• Python and Flask provide the web application and processing endpoints.
• OpenCV 5 handles image-quality analysis, edge and contour detection, perspective correction, and contrast enhancement.
• Tesseract OCR extracts text from images.
• A decision flow provides guidance for conditions such as darkness, blur, glare, and uncertain text.
• AWS EC2 is used to host the application, with a health endpoint to check the service and OpenCV version.

Challenges we ran into

Reflective medicine packaging, small printed expiry dates, stamped ink, and tilted or blurry images make reliable OCR difficult. We also worked through camera permissions, image detection, and deployment differences between the local development environment and AWS.

Accomplishments that we're proud of

We built a working prototype that combines camera guidance, image processing, OCR, and uncertainty handling in one web interface. We also improved the package-detection pipeline and added OCR attempts to re-read uncertain words.

What we learned

OCR confidence is not the same as correctness. Accessibility tools need to communicate uncertainty clearly, guide users toward better images, and avoid guessing when text cannot be read reliably.

What's next for VisionGuide

Next steps include broader evaluation across medicine and food packaging, improved expiry-date recognition, additional language support, better glare handling, and usability testing with people with low vision. Medicine names, expiry dates, and instructions should always be verified when the result is uncertain.

Built With

• aws-ec2
• computer-vision
• css
• flask
• html
• javascript
• opencv
• python
• tesseract-ocr