Shadow

Inspiration

Our vision for the Shadow project was founded on bridging the gap between communication using both hardware and software to its fullest potential. Seamless control of devices via a non-physical medium led to the idea that a software like this needs ASL integration in the real world. This led to the software we created being the voice for ASL. If scaled, the translation could be an immediate and seamless solution for a huge communication gap for everyone. The goal is for communication to be accessible for all.

What it does

The project we created will translate ASL caught on a webcam to text. After the text is translated, it will be turned into audio. Additionally, it lets regular users prompt the device and control it non-physically. With using the webcam, ASL will be translated to audio to provide a seamless translation between people for improved communication.

How we built it

We tried many things before determining what would work. We built the webcam ASL translator and the ElevenLabs text to speech feature. The problem left halfway through the night was getting the API to translate. At 8:20AM we clicked run on a new version using Open Router API instead of Gemini and it finally worked! Through trial and error, we connected several methods of communication together to create this final product.

Challenges we ran into

We ended up needing to do a full pivot on our original idea, as well as running into AI software issues when attempting to build our project. We were able to overcome these challenges and complete this project through determining which tools were most efficient and aligned with our project. With the help of the webcam hardware and ElevenLabs, we were able to create this accessible product to better improve communication between individuals.

Accomplishments that we're proud of

We are very proud of overcoming our unexpected struggles with programming and AI. Having something to submit that works seemed like an impossible goal, yet the hard work has paid off in the end. The rewarding experience of creating the Shadow project despite our challenges is the accomplishment we are most proud of, and will always remember.

What we learned

We learned that we should never give up, no matter how late it may seem. We also learned that our hard work created a project that could help real-life users communicate from ASL to English speech. Each of us expanded off of our creative innovations.

What's next for Shadow

Shadow will be the bridge that allows ASL users to communicate audibly.

Shadow will make it easier for ASL individuals (Students) at universities to communicate and give speeches. Shadow will also equip anyone to able to operate a device based only gestures via a non-physical medium.

Shadow can also have a security interface that would seamlessly and stealthily detect social cues, gestures and signs of aggression to notify security or police personnel.

Shadow can also help with military-stealth drills underwater exploration