Beyond Sight

Beyond Sight is an app designed to allow visually impaired individuals both visualize and navigate the world around them and change how they view the world around them.

Comment

Inspiration

Globally, over 2.2 billion individuals are affected by some form of vision impairment, with around 36 to 39 million being completely blind. In the US, over 12 million adults aged 40+ have vision impairments, including around 1 million classified as legally blind. My app has a large impact, aiding the daily lives of millions and billions of individuals worldwide.

What it does

Beyond Sight has 2 features: the first feature uses ChatGPT and the ProCamera extension to take a picture and describe it, and the second feature uses Google Maps to navigate the users to their desired location.

How we built it

I used MIT App Inventor 2 Block Code as the programming language. I also used the Pro Camera extension, ChatGPT 4.0 APIs, and Google Maps APIs.

Challenges we ran into

In the first function, the user says "what is in front of me?" the app takes a picture which is later verbally described to the user. A challenge with this function was that the camera component of MIT App Inventor could not take a picture by itself when the user asked "what is in front of me". After researching, I found the Pro Camera extension which solved this issue.

Accomplishments that we're proud of

Before coding the app, I was unsure of how to use Chat GPT to describe the image provided but after doing some research on the usage of Chat GPT in MIT App Inventor and the built-in chatbot, I found a solution. I am also proud of the fact that the app is mostly voice-controlled. The only part requiring user interaction is in the Google Maps navigation part as the user needs to first select their specific destination, then press "start".

What we learned

Previously, I hadn't used the Chat GPT API much nor did I use many extensions. The overall process of creating this app gave me experience and knowledge in both APIs and extensions.

What's next for Beyond Sight

In the future, I plan to participate in team competitions with an improved Beyond Sight. I plan on adding a few features. In the first feature, users can use their voice to text others. In the second feature, users can email others. Both of these functions allow users to socialize with others better as they can talk both in person and online.

Built With

Share this project:

Updates