EyePi

Eyes for the Blind

Comment

Inspiration

We're a creative and intelligent group of individuals who have a strong desire to provide solutions through code. Building EyePi allowed us to push ourselves beyond our known capabilities in order to provide a creative solution to urban and social sustainability.

What it does

Combining powerful hardware with versatile software, EyePi provides a creative solution for individuals suffering from visual impairment disabilities, creating a more sure-footed and sustainable future.

A VIRTUAL SOLUTION TO A REAL PROBLEM

EyePi employs a Raspberry Pi 5 as the heart and brains our object-recognition system, capable of analyzing image frames every 3 milliseconds. EyePi will be available as a chest-mounted device, preventing restricted range of motion.

The user turns on the device and through the speaker it lists 6 different language settings for the program. Once the user says their language of choice it turns the camera on and starts detecting objects in its surrounding. Based on the language chosen it will speak out the objects detected in that language.

How we built it

Hardware

   We built the hardware system using a Rasberry pi 5. We added an active cooler, M.2 HAT+, Hailo AI module to have our complete system so that the Pi can be ran efficiently and with proper temperatures. We had to flash the Pi operating system on a SD card and and then connect the Pi to Wifi. We remote SSH to the Pi so that our coding environment was easy to use and implement. We connected VS code IDE as our coding environment and using linux commands to create a virtual environment, flash libraries, run scripts, and overall debugging. We also connected a portable camera to our pi and a speaker so that the Pi can take in video feed and output the objects in the video frame to the speaker.

Software

 We built our software system which comprised of the objection detection and classification, and speech processing and output. For the object detection and classification we used the YOLOv8 computer vision machine learning model as it was trained on thousands of images and has around 80 labels. We used the OpenCV library to process the frames that initially came from out laptop cameras and moved to using a portable camera. each frame was processed through the YOLOv8 model and it outputs all the objects in each frame and gives details on each such as the bounding box around it, the label itself, and the accuracy of the detection of that label.

We also created other different scripts for the language processing so that when the program starts it asks the user for the language setting. We used the speech recognition and Google translate python library to convert the audio into text, and text back to audio, as well as use google translate to do have our different language options.

Challenges we ran into

The biggest challenge we ran into was remote booting and developing with the Raspberry Pi OS. We discovered the issue by reading the network IP Addresses and seeing that they were being scrambled due to the school WiFi's security protocols We overcame this by switching to hotspot and using micro-HDMI instead of remote SSH.

Another challenge we faced was audibly sending and receiving the verbal messages between the Pi and the user. Our solution to this was communicating between the input and output devices (microphones and speakers) in order to prevent overlap.

Accomplishments that we're proud of

During the course of this project, we overcame a variety of hurdles. One of our proudest moments was fully transferring our codebase from our local machines to the Raspberry Pi OS, consistently encountering and resolving integration issues the whole way through.

What we learned

We learned the importance of persistence. Without it, we wouldn't have made it through the first night. Persistence proved to be a common theme throughout the entire challenge. The countless hours researching new tech, testing existing code, troubleshooting new problems, and designing better algorithms strengthen us and gave us a challenge worth fighting for.

What's next for EyePi

The modular nature of Raspberry Pi allows us to aim high and further develop the device to create a wearable product, capable of incredibly fast image display and classification. Additionally, an updated and enlarged list of classifiers would further fine-tune our precise algorithm. Increased processing power would further create a more realistic and professional experience for the user.

Share this project:

Updates