-
-
Home page
-
Sign Language recording Page
Inspiration We have one team member whose family member is deaf, and we learned more about the deaf culture through them. To most people, American Sign Language (ASL) is not a tool, but a culture, a part of identity itself. That's why we built something that celebrates and respects that depth. We aimed to offer a way for fingerspelling—the ASL alphabet—to be viewed and understood using technology, not to replace communication but to bridge the gap for those who are not yet able to understand it.
What It Does Fingual is a program that takes video input of one's fingerspelling and generates real-time text output. It utilizes OpenCV to track and label hand keypoints, a Convolutional Neural Network (CNN) for hand sign classification, and a generative AI layer to optimize the analysis for faster, more accurate prediction.
How We Built It We created Fingual from scratch with a mix of tools. Finger detection and tracking were done using OpenCV. The front-end was built in Xcode to make it have a good mobile experience, and the backend runs with Flask for video processing and AI inference. Our AI model was trained with TensorFlow/Keras from data we manually curated.
Challenges We Ran Into We're newbs, so just building a full-stack AI app was a learning experience. The hardest part, hands down, was obtaining good, usable ASL fingerspelling data. Our datasets were mostly incomplete or inconsistent. We shifted gears and built a mini dataset from scratch by filming, annotating, and training ourselves.
Accomplishments That We're Proud Of We're glad we didn't give up when we were stuck on the data problem. We came together, took sign readings ourselves, and built a functional model. It was like we had a mini-coding problem within our whole project, and solving it was a big success for us as a group.
What We Learned We learned about how to combine a front-end and back-end system, how to train and test AI models, and how important clean and well-formatted data is in machine learning. Most importantly, we learned how to be agile and work together under pressure.
What's Next for Fingual We plan to expand Fingual from ASL fingerspelling to other sign languages like BSL (British Sign Language) and LSF (French Sign Language). We also plan to increase accuracy, mobile-performance optimize, and eventually extend to full-word or sentence recognition—not just the alphabet.
Log in or sign up for Devpost to join the conversation.