Safe Step

Image of the frontend before a request is sent. Showing the map and the dashboard saying that there are no active requests.
Image of the frontend after a user has requested help. Showing the dashboard with a button to find the location of the user in need of help.

Inspiration

We decided that this hackathon, we wanted to build something to help people in need. We've decided that helping people with disabilities is less common in big tech companies as there won't be much profit. With the evolution of AI technologies, we know that a lot more things is possible and so we decided to explore the idea of helping people with visual impairment. Using AI technologies, we can help them live safely and bring a little vision to their life.

What it does

Our application is divided into 2 parts;

Vision and Speech
Helping Those in needs Our application is always listening and watching. The user can ask the AI through it's keyword ("Jarvis") and asks it to describe what is infront of him or to ask for help. In the first case, the AI will describe what is in front of him and suggest appropriate direction. In the second case, it will tell him that help has been notified bringing us to the second part of the application. We made an Website that shows if there are people in need of help on a map. This way, a nearby volunteer can come to aid the user. This works in real time right when asked by the user.

How we built it

We decided to use Python for the backend, using both Gemini and ElevenLabs to describe the scene and to communicate it to the user.

For the Safe-Step platform, we used basic html hosted on Github Pages with a .tech domain. For the real time notification of request for help, we used a websocket using JavaScript and an Python FastAPI.

Challenges we ran into

For all of us, it was our first time using AI's API. We've all used AI tools, but we've never linked it directly into our code. This was a learning curve that we successfuly accomplished.

Accomplishments that we're proud of

We are really proud to have a real time dashboard of people requesting help as well as having the AI describe what it's seeing to the user. We feel like this project could really be helpful to people with visual impairment.

What we learned

We learned a lot of things such as: how to setup a domain dns, using AI APIs, linking all of the projects part together smoothly, working on a project with a limited time and a lot more.

What's next for Safe Step

We also had plans for the AI to tell the user if he notices that there is a level of danger ahead that the user needs to be aware of. We kept this feature as an extra, since the hackathon is of short duration. Though many little improvments and new features can be added to make this application much more helpful.