ClearVue: Redefining Sight, One Hat at a Time

Inspiration

Globally, an estimated 40 to 45 million people are blind and 135 million have low vision. Utilizing the GPT-4 AI model and advanced sensory technology, our mission is to help these individuals regain the ability to live thier every-day lives.

What it does

Four Modes

In Front: Provides a short real-time auditory feedback, enabling users to identify objects in their immediate environment.

Reading Mode: Narrates labels and text that is positioned in the camera frame.

Story Mode: Creates a narrative of the user's environment, providing a comprehensive understanding of their surroundings.

Object Detection: Actively monitors the camera feed for object in frame (ex. laptop, bottle, teddy bear, etc.)

How we built it

Hardware

We used a Raspberry Pi to execute our project, running on a small, portable battery pack. A Raspberry Pi Camera was implemented to meet our vision capabilities. A small breadboard and push button provide a user interface, pairing with a bluetooth audio device (speaker, headphones, earphones, etc.). A stylish hat is used as the vessel for all of the components.

Software

In the first three features: In Front, Reading Mode, and Story Mode, we used OpenAI Vision to describe the image before passing it to a text to speech model to transform the text into words. Our final feature, Object Detection Mode, utilizes local computation to run a small model that actively classfies what is in front of it. This primiarly utilizes TensorFlow and is incorpated seamlessly into the first three. To cap everything off, our landing page was created using NextJS. Its amazing design shows the true power of what this product has to offer.

Challenges we ran into

One of our largest struggles was integrating our continuous object detection mode and description modes. Originally running as two separate scripts, combining these functionalities gave us some trouble, but we learned a lot while solving these issues.

Accomplishments that we're proud of

We were happy to develop a lightweight, working product, that can be used for hours on end to help those who are visually impaired.

What we learned

Working with Raspberry Pi and integrating all the necessary hardware, software, and libraries needed to tie everything in together.

What's next for us

Whatever opportunity presents itself!

Built With

• nextjs
• openai
• python
• raspberry-pi
• tailwindcss
• tensorflow