Assistive AI

Inspiration

Help people with vision impairments to be able to enjoy websites and social media. Currently the alt Attribute in HTML <img> tags is very limited and depends on developers to put in detailed information describing the image to help those with disabilities.

What it does

Uses AI to describe the images/websites/social media posts in detail

How we built it

Using PartyRock. However, the system is limited in accessing pictures on the websites.

What's next for Assistive AI

1. I am going to create a browser plug-in that is going to use the bedrock API to be able to access the images on the websites
2. Next I am going to create a version that will be run on an Arduino or Raspberry pi connected to a camera and glasses. This setup will be connected via bluetooth to a smartphone. This prototype will be tested in the real world, where a user can wear the glasses and with the press of a button or voice assistant, ask "what am I looking at" and the camera will take a picture and run it through Claude vision to then respond to the user with detailed information about their environment. This can be used to help those with visual impairments or people that are blind to navigate the world in a safer way and enjoy the world around them as well.

Built With

• bedrock
• vision