Vision Smart Device

Empowering vision beyond sight

Comment

Inspiration

"This all began with Charity."

Charity is a friend and fellow student in my campus fellowship. She lives with glaucoma-induced blindness. After our weekly church services, I often saw her sitting and waiting—sometimes for over an hour—for someone to help her walk back to her hostel. Friends meant well, but they were also busy, attending other meetings or programs.

I couldn’t shake what I felt the day I asked her, “How do you usually get home when no one is free?” and she simply smiled, “I wait.” That moment turned my empathy into engineering, seeing I was a computer engineering major. I knew something had to be done.

So, I started researching.

Most of the existing smart assistive solutions I found—like smart glasses from Meta or advanced Chinese wearables— were extremely expensive and reliant on internet connectivity. Not only were they far out of reach for students like Charity, but they also assumed always-on Wi-Fi or cloud processing—which just isn’t feasible for everyday users in Nigeria.

Across many African cities and campuses, visually impaired students face daily mobility barriers. While white canes help with immediate obstacles, they can't detect overhead objects, moving vehicles, or provide directional navigation. Guide dogs are virtually non-existent here, and human assistance—while kind—is not always reliable or sustainable.

The core challenge is mobility without dependency.

So I built the Vision Smart Device: a wearable, AI-powered, voice-guided system that enables visually impaired users to detect objects, avoid obstacles, and navigate their environment independently, using offline AI processing and a GPS system localized to familiar spaces like the University of Ilorin campus.

"Charity shouldn't have to wait to go home. And neither should anyone else."

What it does

The Vision Smart Device is a wearable assistive system designed to empower visually impaired individuals with more independence in their daily movement. Worn like a pair of glasses, it combines real-time object detection, obstacle avoidance, and offline voice feedback to help users “see” through sound.

Using a Raspberry Pi 4 and a custom-trained AI model, the device detects objects like people, vehicles, and other environmental hazards in real time. These objects are announced to the user using text-to-speech, ensuring they are aware of what’s in their path. To handle close-range threats, the system also uses ultrasonic sensors to detect obstacles that may not be visible to the camera—such as glass doors, stair edges, or narrow walkways. If an obstacle is detected within a meter, the user is alerted immediately with an audio warning.

I am also working on integrating offline GPS-based navigation into the device. I've already downloaded the offline map of the University of Ilorin campus, connected a GPS module, and tested the route logic in a desktop simulation environment. The goal is to allow users to speak a destination (like “Female Hostel”) and receive step-by-step voice directions to get there—without any internet connection. This part is still under development, but core components like offline routing and real-time location tracking are already functional in testing.

In summary, the Vision Smart Device provides essential environmental awareness and aims to enable independent navigation, especially in contexts where conventional solutions are either unavailable or unaffordable.

How we built it

We used a Raspberry Pi 4 as the main processing unit, integrating a Pi Camera for visual input and ultrasonic sensors (HC-SR04) for close-range obstacle detection. All components were connected through GPIO pins and powered via a 10,000mAh USB power bank, allowing over 4 hours of mobile use. A custom-trained YOLOv5-nano model was optimized using Roboflow and deployed directly to the Pi, enabling offline real-time object detection. The camera captures live video, detects objects like people or vehicles, and announces them via text-to-speech using the eSpeak engine. For navigation, we used a NEO-6M GPS module and downloaded offline maps of the University of Ilorin from OpenStreetMap. Route simulations were tested successfully on a PC, and hardware integration is in progress. The device was fully developed using Python, with supporting libraries including OpenCV, PyTorch, and RPi.GPIO. The final prototype is assembled in a wearable glasses format.

Challenges we ran into

To ensure smooth performance on a 4GB Raspberry Pi 4, we optimized our system using a lightweight YOLOv5-nano model and limited detections to three objects per frame to maintain real-time responsiveness. We faced early setbacks when two Pi Cameras failed, leading us to switch to a USB webcam, which, although limited in resolution and field of view, kept the project moving. Budget constraints prevented us from using higher-spec devices like the Pi 5 or 8GB models, so we had to carefully balance speed and accuracy. Our initial goal of building smart glasses had to be adjusted due to hardware bulk, resulting in a wearable format that remains comfortable. To keep the device lightweight, we powered it with a 10,000mAh power bank, easily pocketable and offering 3–4 hours of use. Arranging components—camera placement, sensor wiring, GPIO pin access—was another challenge, especially within the limited space of a wearable setup. We also faced minor system lag, but chose to prioritize accurate feedback over faster but inconsistent responses. Lastly, tuning the audio output logic—especially distinguishing between object labels and distance alerts—required careful refinement to ensure clarity for the user.

Accomplishments that we're proud of

We successfully deployed an offline object detection system using YOLOv5-nano on a Raspberry Pi 4, enabling real-time identification of objects without internet access. The device uses local text-to-speech (eSpeak) to announce detected objects and obstacle distances, giving users a clear sense of their surroundings through audio. By combining input from both a camera and ultrasonic sensors, we achieved reliable detection at different ranges—something rarely found in low-cost solutions. Despite limited resources, we built the entire system with affordable, locally available components, adapting the original smart glasses idea into a lightweight wearable format that remains practical and comfortable. Although full GPS navigation is still being integrated, offline route simulation using OpenStreetMap already works as expected. Every part of this project was built with the real needs of African users in mind—simple, offline, affordable, and effective.

What we learned

Through this project, we learned that building assistive technology is not just about innovation—it’s about empathy, simplicity, and real-world usability. We realized that even powerful AI models must be optimized for limited hardware when affordability is a goal. We also learned to adapt quickly when original plans, like using a Pi Camera or smart glasses format, didn’t work out. More importantly, we gained hands-on experience with system integration—combining computer vision, audio feedback, and hardware sensors into one offline device. Finally, working with real user scenarios reminded us that small design choices—like where a camera is placed or how audio is delivered—can make a big difference in someone’s daily life.

What's next for Vision Smart Device

The next step is to fully integrate offline GPS navigation into the wearable system. We've already tested the route logic and offline maps on a PC; the goal is to make voice-guided navigation fully functional on the Raspberry Pi. We also plan to refine the hardware design into a more compact and user-friendly wearable—ideally returning to the original vision of smart glasses as smaller components become accessible. Another priority is conducting real-world testing with visually impaired users like Charity, to gather feedback and improve usability. In the long term, we aim to open-source the design, optimize for solar-powered use, and explore mass production partnerships to make the device available to schools, NGOs, and individuals across Africa who need it most.

Built With

Share this project:

Updates