SignTence

Inspiration

SignTence was motivated by a personal interest in learning American Sign Language (ASL) and exploring how computer vision can be applied to facilitate accessible communication.

What it does

The application processes real-time webcam video to detect ASL gestures, translating those visual inputs into coherent, text-based sentences.

How we built it

The core system is developed in Python. We trained and deployed a YOLOv11 object detection model to perform real-time image recognition on live video feeds.

Challenges we ran into

The primary technical hurdle was training the computer vision model, specifically formatting the datasets and optimizing the model's parameters to reliably identify gestures.

Accomplishments that we're proud of

We successfully bridged the pipeline from training custom machine learning models to executing accurate, real-time gesture recognition through a standard webcam.

What we learned

The development process provided practical experience in training object detection models, managing custom model weights, and integrating LLaMA for downstream natural language processing.