Inspiration
Millions of deaf and hard-of-hearing people in Mexico rely on Mexican Sign Language (LSM) to communicate, yet most of the hearing population does not understand it. This creates significant barriers in everyday situations, education, and healthcare. Signal Glove was born from the desire to bridge this communication gap using accessible hardware and artificial intelligence.
What it does
Signal Glove is a smart wearable device that captures hand gestures and finger movements in real time, translating Mexican Sign Language (LSM) into readable text and spoken audio through a connected application. This enables fluid, two-way communication between LSM users and non-signers.
How we built it
- Hardware & Sensors: Designed an ergonomic glove integrating flex sensors and an inertial measurement unit (IMU) connected to an
[ESP32 / Arduino]microcontroller to capture hand posture, finger flexion, and spatial orientation. - Prototyping: Modeled and 3D-printed custom enclosures for the electronics to keep the wearable lightweight and comfortable.
- AI & Software: Trained a machine learning classification model in Python to recognize LSM signs from sensor data streams, paired with a real-time interface that converts predictions into text and text-to-speech output.
Challenges we ran into
Calibrating flex sensors to adapt to different hand sizes while minimizing signal noise during fast transitions between signs was a major technical hurdle. Optimizing the inference pipeline to keep latency low enough for natural conversation also required careful tuning of both firmware and data filtering.
Accomplishments that we're proud of
- Built a fully functional wearable prototype capable of translating LSM in real time.
- Awarded National Winner of the James Dyson Award México 2024 for engineering design and social impact.
- Showcased the prototype live at Maker Faire CDMX 2024, receiving direct feedback from the community and makers.
What we learned
Developing assistive technology requires designing with users, not just for them. Balancing hardware ergonomics, sensor precision, and real-time software performance taught us how to iterate rapidly from early breadboard tests to an award-winning physical product.
What's next for Signal Glove
Expanding the vocabulary to recognize continuous dynamic sentences rather than isolated signs, miniaturizing the PCB for greater daily comfort, and continuing to collaborate with the deaf community in Mexico to refine usability.
Built With
- 3d-printing-software-/-ia-/-app:-python
- c++
- esp32
- flutter-(o-react-/-javascript
- machine-learning
- sensors
- tensorflow
