ESE 1110, Team 12, Naviglass

Imagine navigating the bustling city of Philadelphia without sight, every step uncertain. Naviglass is a pair of smart glasses with AI detection technology to help users navigate safer and better.

Comment

Inspiration

Our world today is heavily modernized and living in a vibrant city like Philadelphia currently, we realized the hidden struggles that the visually impaired face while navigating our world. Naviglass was created to address the challenges of spatial awareness and safe navigation, using AI and haptic feedback. The goal was to build a cost-efficient and comfortable wearable device that uses computer vision to provide smart, conversational assistance for detecting objects in real time.

What it does

💡 Core Functionality

  • Object Detection: Uses the Ultralytics YOLO11n model to process input from the Raspberry Pi Camera and identifies objects (e.g., trees, stop signs, cars) in real time.

  • Haptic Feedback: When an object is detected and is nearby, six vibration motors activate at varying intensities based on object proximity, providing spatial awareness. Once the object is "locked in," only the motors continue to activate, and the narration does not repeat.

  • Smart Navigation: Integrates Text-to-Speech (TTS, using Piper Amy Low Quality) to provide a calming, conversational voice assistant that alerts the user to detected objects.

  • Instant Connectivity: The device features a custom Bluetooth Manager hosted on a web server that allows users to quickly pair, connect, and disconnect any audio device for the TTS output.

⚙️ Hardware and Interface

  • The system uses a Raspberry Pi 4, a Pi Camera, two Ultrasonic Distance Sensors, and six Vibration Motors.

  • The user interface is a simple web page that displays the live camera feed and features a "Scan for Devices" button to manage Bluetooth connections.

  • The final prototype enclosure was designed using SOLIDWORKS CAD and fabricated via 3D printing/laser-cutting, utilizing an elastic band to ensure the device remains comfortable and wearable for extended periods.

Challenges we ran into

Building Naviglass required overcoming significant hardware and software challenges. We faced tight spatial constraints fitting a Raspberry Pi 4 and sensors into a wearable enclosure without a custom PCB, requiring multiple CAD iterations for fit and comfort. Electrical challenges included complex soldering for six vibration motors and managing critical component failures like overheating power banks. On the software side, we developed custom classes to handle asynchronous Bluetooth and TTS interactions, while heavily optimizing the YOLO11n model to run efficiently on the Pi’s limited CPU.

What we learned

Developing Naviglass provided us full-stack systems engineering experience. We learned practical circuitry skills wiring and soldering vibration motors and ultrasonic sensors to the Raspberry Pi 4. Our AI expertise grew by deploying the Ultralytics YOLO11n model for real-time object detection, integrated with Piper Amy Text-to-Speech. We also enhanced our IoT capabilities by building a web server and custom Bluetooth manager, and refined our prototyping with SOLIDWORKS CAD, 3D printing, and laser cutting.

Built With

Share this project:

Updates