Too Long Didn't Read: A guide to instructions

Home Page
Text Page
TTS Generated Response with subtitles (needs text to be present)

✨✨ Our Inspiration!!

My grandmother (Ayush's) was recently sick due to taking the wrong medication. This is due to labels' abhorrent and cluttered nature causing a lapse in judgment and knowing this could have been avoided. We decided to create something that would prevent this from happening to others and ensure that we lazy folks (or those with visual impairment), regardless of age or ability, have the ability to get past those huge walls of text to receive a Too Long Didn't Read (get the name haha…) :get_out_of_here:

🙌 How does TLDR: A guide to instructions solve the problem?

Normally what do people do when they are met with a huge block of text?

A normal person would just scoff at it and move on right? (maybe you are a psycho and read all of it who knows) but I am sure everyone would benefit by saving those good few minutes.

By analyzing the piece of text, whether it be medication or just a textbook, TLDR takes the information and returns the key points of it, allowing you to ask questions to better understand the product.

🧑‍💻🧑‍💻 How we Built it!

The backbone of the project was built on OpenCV which allowed us to use computer vision at the tip of our fingers.

Using YOLOv8, a model trained to detect objects, the video is able to outline and keep track of various objects, and to load it on screen we used SuperVision to keep track of the items. Paired with Tesseract OCR, once the user wants to capture, we take a screenshot of the image and analyze the text.

Tesseract OCR is an optical character recognition engine. To accurately comprehend the picture, we must provide a smoothing, grayscale, change in dpi, and much more to the image.

Next, we transfer the text and allow the Generative AI to extract the information and output the key details. Finally, we use GTTS and speech_recognition to take outputs and inputs respectively.

😰😰😰Challenges we ran into

We ran into many challenges to name a few: Issues with implementing several different files and ideas Trying to optimize the detection of Tesseract Getting the generative AI to recall past data Credit card information not loading properly 😔

😤 Accomplishments that we're proud of

Finishing a hackathon…

🤓 What we learned

We learned how to use Tesseract engine and how to optimize it How to implement generative AI into our projects How to use TTS and speech recognition

🔜🔜🔜 What's next for TLDR: A guide to instructions

Currently, we are only limited to computers and laptops which is a big hindrance for a quick situation such as this one so our next best approach would be to branch out.

Secondly, we would like to have more accurate readings, so training our own AI would be the next step.