BlindSight

• 

  BlindSight

• 

  Obstacle Detection

• 

  Obstacle Detection

• 

  Ray Tracing

• 

  Ray Tracing

• 

  Ray Tracing

• 

  Audio Output

• 

  Pipeline

• 

  ToDo

Inspiration

The inspiration behind our project, BlindSight, was to address the challenges faced by the 1.1 billion visually impaired and 36 million blind individuals worldwide. Our goal was to utilize sensor technology to enhance their safety and understanding of the world around them, ultimately improving their quality of life. By promoting inclusivity and social sustainability, our project aimed to empower this marginalized community.

What it does

BlindSight is a system that utilizes a Time-of-Flight (TOF) sensor to analyze the surrounding environment. It employs object recognition techniques to detect objects within the environment and provides audio feedback to the user based on the distance to these objects.

The system consists of a module mounted on the user's chest and headphones worn by the user for audio output. As the user moves within a closed area, such as a building, the module continuously analyzes the surroundings. In situations where the user is on the verge of colliding with an object (within a distance of less than 1 meter), the system emits a high-intensity alarm sound as a warning signal. For non-critical scenarios (within a distance of 1-3 meters), the system provides low-intensity audio information.

How we built it

To provide a more technical explanation of the steps involved in developing BlindSight, here is a breakdown of each stage:

Video Processing: We utilized a camera to capture video frames in real-time. Each frame was processed independently to extract relevant information for obstacle detection.

Floor Area Calculation: To distinguish between obstacles and the floor, we performed floor area calculation within each frame. This involved analyzing the depth information obtained from the Time-of-Flight (TOF) sensor to identify the region representing the floor.

Matrix Projection and Division: The processed frames were projected onto a matrix for further analysis. The matrix was divided from the middle, enabling separate processing of the left and right sides. This division allowed us to handle the different spatial characteristics of the scene and focus on specific areas of interest.

Audio Output: We leveraged the Python library "sounddevice" to generate audio feedback for the user. The audio output was based on the analysis of the left and right sides of the divided matrix. By comparing the depth values and spatial features of objects on each side, we determined the appropriate audio intensity and cues to convey to the user.

Continuous Loop: The entire pipeline, consisting of video processing, floor area calculation, matrix projection and division, and audio output, was organized within a continuous loop. This loop ensured real-time processing of the video feed, allowing for immediate obstacle detection and audio feedback.

Although we successfully implemented frame processing and obstacle detection, the integration of object recognition into the pipeline is still a work in progress. Once integrated, it will enhance BlindSight's ability to recognize and classify objects within the environment, providing even more valuable information to visually impaired individuals.

Challenges we ran into

During the development process, we encountered several challenges. Fine-tuning the algorithms for accurate object detection and classification in various environments, including calculating the floor area, proved to be a significant challenge. Additionally, optimizing the system's performance to provide real-time audio feedback while minimizing latency required extensive experimentation and iteration.

Accomplishments that we're proud of

We take pride in successfully developing a functional system that leverages the TOF sensor for environment analysis and obstacle detection. BlindSight effectively provides audio feedback to visually impaired individuals based on the distance to detected objects, enhancing their safety and awareness. We have also achieved reliable performance in critical situations by providing timely alarm sounds.

What we learned

Throughout this project, we gained valuable insights into sensor integration, object recognition algorithms, and real-time audio feedback systems. We expanded our knowledge of accessibility technologies and deepened our understanding of the challenges faced by visually impaired individuals. Additionally, we developed troubleshooting and optimization skills to enhance system performance.

What's next for BlindSight

Looking ahead, we have several plans for the future of BlindSight. We aim to further extend the system by integrating object recognition capabilities. Additionally, we see potential applications for BlindSight in assisting blind animals and providing them with improved navigation and safety. Moreover, we envision implementing BlindSight in museums to offer enhanced guidance and accessibility for visitors with visual impairments. We will continue refining and expanding the capabilities of BlindSight through collaborations and partnerships, seeking to advance the impact of this technology.

Built With

• python
• yolo