SeñasVision

Inspiration

As the founders of SeñasVision, we all know someone who struggles with hard-of-hearing. Specifically, one of the founders has a story that really touched home, and overall inspired us to continue with this project. Nataly has a cousin whom she grew up with in Mexico, and saw his and those around him struggling to communicate with one another. Therefore, we noticed this problem and decided to make a web app that will bridge this communication gap.

What it does

Our web app is designed to have a home page, and a few navigation bars. The home explains what SeñasVision is, and its relevance. The navigation bars include an about us statement, and a resource tab to learn more about Lengua de Señas Española (LSE). Within the home page, there's an option to start translating, and it will transfer you to a page to enable your webcam. From there the camera will start following your hand movement and detect LSE. It will then notify you what that hand gesture means in the Lengua de Señas Española alphabet system.

How we built it

We built this using React, CoLab, Kaggle Datasets, Python, and Computer Vision. This project uses the React framework to simplify our web dev process. We uploaded an LSE dataset from Kaggle to Colab in order to train the data to implement a model. Thus, whenever you do a gesture at the camera, it's able to detect what letter from the alphabet it is.

Challenges we ran into

We ran into many challenges when creating our web app, especially since this was our first ever collaboration project and Hackathon. Although GitHub may seem self-explanatory to most people, we found it quite difficult not only cause it's our first collaboration project, but we were under a time constraint. We also didn't know the term of a virtual environment until our mentor explained the importance. Another difficulty was setting up our data to start training, we weren't familiar with how to start, but our mentor gave us a great recommendation. In the beginning, we wanted to use a Streamlit framework but at the end switched to React.

Accomplishments that we're proud of

1.) Focused on learning rather than winning. We weren't afraid to seek help! 2.) Getting to create a model from a dataset 3.) Applying computer vision to our project

What we learned

1.) Learned how to collaborate with others on GitHub and VS Code 2.) Learned the difference between a virtual and global environment, and the importance of maintaining a virtual environment while processes are being developed 3.) What types of frameworks there are, and why they are used. 4.) How to train a dataset and open a computer vision source in Python 5.) Made a part of the dataset by taking our own pictures.