StemSigns

Inspiration

Due to its relative youth and widely dispersed user base, American Sign Language (ASL) has never developed standardized vocabulary for the many terms that have arisen in advanced Science, Technology, Engineering, and Mathematics (STEM) fields. This makes it hard for deaf students to learn in their native language, and it makes communication between both deaf and hard of hearing scientists and engineers far more difficult.
As any intro linguistics student could tell you, language use and evolution cannot be dictated by the few, no matter their expertise. This approach has failed time and again throughout history, and, no doubt, this case will be no exception. Instead, languages change because their users choose to change them.
This project is an attempt to connect ASL users together, so that we can introduce the necessary vocabulary to the language. This will make it much easier for those in the Deaf community to pursue careers in technical fields.

Online education with a reliance on un-captioned videos or audio hurts the deaf community
Zoom calls and webcasts are very difficult to effectively sign over
This hurts the ability for some students to learn in their native language
Our app enables classes to share and develop vocabulary *Enabling people to communicate and develop signs about new terms, such as “COVID-19”
Empowers communication about the newest terms whose signs haven’t developed yet
A global, online resource for language development
Many existing audio-reliant resources are not designed or accessible to the deaf community making the transition all the more difficult
Our app strives to provide more online and accessible resources to the ASL community in this time of need

StemSigns serves as a visual dictionary of STEM terms for the deaf community
We provide an existing database of short videos demonstrating STEM terms via ASL
The community is also encouraged to submit their own videos providing possible ASL translations for words
These new videos will join the StemSigns database and help broaden / expand ASL
We also provide an option to receive daily text updates to encourage the learning of a new word a day
StemSigns also keeps a hierarchical list of categories and related words, allowing for easy navigation and expanding of one’s own vocabulary

The app is built in react.js as a website (with the requisite HTML/CSS)
We used GCP for user authentication/sign-in and Firebase for the word/definition database
Twilio enables us to send Words of the Day to users, encouraging people to keep learning new things!
We have a MongoDB Atlas backend for our user/phone number database
Youtube’s Data API (GCP) was used to allow for instant user uploads to a community account
HTML5 Webcam support
UIPath for error detection and early warning
Material.ui for the UI/styling

Took a lot of time to figure out OAuth authentication for Youtube uploads
The dataflow from Webcam->Local->Youtube->Backend was decently complex/tricky
Styling was a bit of a challenge since we had no one with that expertise on our team
The Database/Backend took a bit of time as we were having trouble having the twilio side working

WinHacks 2020
Hack Quarantine
Hack Kosice Digital
COVID-19 Global Hackathon 1.0
LA Hacks 2020
- Winner [Twilio] Best use of Twilio API
- Winner [MLH] Best UiPath Automation Hack
- Winner [PwC] Honorable Mention Journey to the Future
HackCOVID 2
- Winner Champions of Empowerment

I worked on the Youtube Data API for video uploads. Beyond this I worked on a lot of branding, presentation, demos, and documentation.

Ray Altenberg
I worked on creating the core React app, authentication, YouTube uploads, Firebase integration, and the search functionality

Bryan Lim
I worked on integrating Twilio with MongoDB to send useful daily ASL information to our users

Alan Brilliant
I worked on implementing the webcam functionality of the website, helped develop the front end in React and setup MongoDB. I also edited the video for our presentation.

Farhan Saeed