Inspiration: My current work at SLU Chrome Lab has instilled in me a passion for building accessible resources to serve the disabled community. With ASLingo, we wanted to specifically serve the deaf community and mute in their struggles to communicate with people and those who want to learn ASL to communicate with them.
What it does: ASLingo is a desktop application with two learning modes: Practice and Test. In Practice mode, users can practice learning sign languages by completing multiple-choice problems with pictures. In Test mode, users can test their knowledge by checking the accuracy of their hand signs with the built-in computer vision detection.
How we built it:
Tech Stack Frontend: Electron.js (HTML, CSS, JS) Backend: Flask + FastAPI (Python) Machine Learning: PyTorch (ResNet18) Other tools: OpenCV, Pygame, Torchvision
Challenges we ran into: Each team member created separate parts of the project, resulting in three mini-projects that were challenging to combine. Thanks to the mentors' help, we finally figured out how to solve this. We also ran into other challenges because all of us were not familiar with the tech stacks. However, we learned and adapted quickly by using online resources.
Accomplishments that we're proud of: We are proud of how we were able to quickly adapt when we ran into the issue of struggling to integrate all of our three mini-project codes. The mentors showed us how to integrate the two more simple landing page and practice page projects, and we were able to rapidly learn the technique we observed and apply it to integrate the more complex testing page project. This was the entire team's first hackathon, yet we were able to work together efficiently to produce a fully functioning desktop app.
What we learned: We learned new tech stacks, especially how to work with APIs. We also made many new connections, especially with peers and mentors who are passionate about the tech industry. We also learned the importance of mentorship and asking for support.
What's next for ASLingo:
- Improving the visual aspects.
- Fine-tuning the American Sign Language Detection model. Aiming to detect words and sentences in real time.
- Adding more types of learning methods and testing methods.
- Adding score tracking for all the methods.
- Adding sound handling.
Log in or sign up for Devpost to join the conversation.