VoiceBridge

AI-Powered Accessibility for All​

Comment

Inspiration

As a team, we strive to develop technology that is not only innovative, but inclusive. Current technology is rarely accessible to people with impairments. We challenged ourselves to develop an innovative tool that enhances independence for users and bridges the gap between technology and accessibility all the while making technology truly inclusive.

What it does

This is just an example of what could be done if the API for the Dain was publicly available, but due to limitations, we can only do certain things through the sandbox. With Dain’s unique blockchain system, which is very secure and a whole new world, users can add “services” that may suit their needs. Whether for stocks, cooking, or delivering food, Our technology combines Dain and the community to make a unique mixture. With Dain’s services, our software is flexible enough to be anything you want and simultaneously be your assistant on your device. You can navigate through your voice, click what you like with a word, control your screen, scroll up and down, and many more exciting features.

How we built it

Attending the Dain Workshop was a big start, we incorporated python with several libraries such as flask, openai and pyautogui with our Dain Service to do impossible things like controlling our computers, communicating with ChatGPT and many more amazing things that we did.

Challenges we ran into

Before the Dain Workshop, we had trouble finding the API keys. We also had issues with Dain's sandbox sometimes not working, and we had to refresh it once in a while.

We were also having a hard time implementing the screenshot function and saving it to one of our temp directories. We also encountered problems with implementing OCR and PyAutoGUI, detecting buttons, text or something that a user would like to click on.

Although these were really hard, we managed to do it and we are proud to make this into a reality.

Accomplishments that we're proud of

We were actually able to emulate key presses, clicks, and mouse movements through Dain's Sandbox. We were also able to integrate other services, and it shows how flexible our service is with other services that Dain offers. If we made this into an software, there are a lot of possibilities that we could do with this one.

What we learned

Team Work and collaboration, Planning, Pyautogui, flask

What's next for VoiceBridge

One of our future goals is to make the software and a Chrome extension to parse the website so that the LLM can fully understand what you are seeing, and what you are doing within the website. Making it much more accessible in the future. Scale in and scale out function and automatically detecting face using CNN

Interested to see what we envisioned? Check out: VoiceBridge | Hacksc 2024 ( End Goal)

Built With

Share this project:

Updates