GloveCom_2.0

GloveCom 2.0 is an AI-powered mobile app that translates sign-language gestures into real-time text and speech, enabling accessible communication through customizable gestures

Comment

Inspiration

Communication can become difficult when people who use sign language interact with people who do not understand it. This inspired us to build GloveCom 2.0, a mobile application designed to make sign-language communication more accessible, natural, and convenient.

Instead of requiring specialized hardware or wearable devices, GloveCom 2.0 uses the smartphone's camera to detect hand signs and convert them into meaningful communication directly through the app.

What it does

GloveCom 2.0 is a complete Flutter-based mobile application that uses the device's camera to detect and recognize hand signs in real time.

The application captures hand gestures through the camera, processes them using computer vision and gesture-recognition techniques, and converts recognized signs into text. The generated text can then be converted into speech, allowing users to communicate more easily with people who may not understand sign language.

The app also provides customizable gestures and messages, allowing users to personalize the way their signs are interpreted.

Key Features

  • Real-time hand-sign detection using the smartphone camera
  • Gesture-to-text translation
  • Text-to-speech conversion
  • Customizable gestures and messages
  • Gesture management
  • Translation history
  • Camera-based interaction
  • User-friendly Flutter interface
  • Real-time recognition experience
  • Cloud-based data integration using Supabase

How we built it

GloveCom 2.0 was developed using Flutter and Dart as a complete mobile application.

The smartphone camera is used as the primary input for capturing hand gestures. Computer vision and hand-tracking techniques are used to identify the user's hand and extract relevant gesture information.

The recognized gestures are mapped to corresponding letters, words, or user-defined messages and displayed as text within the application. The text can then be converted into speech to provide an audio output.

Supabase is used for cloud-based data storage and synchronization, including user-related gesture information and translation history.

The application follows a modular approach, making it possible to improve the recognition system and add additional gestures, languages, and AI capabilities in future versions.

Challenges we faced

One of the major challenges was achieving reliable hand-sign recognition under different lighting conditions, camera angles, hand positions, and backgrounds.

Another challenge was distinguishing between different hand gestures accurately while maintaining a responsive real-time experience on a mobile device.

We also had to handle camera input, gesture processing, application state, translation results, and text-to-speech output while keeping the interface simple and easy to use.

Designing a customizable gesture system was another challenge because users should be able to associate their own gestures with specific messages.

What we learned

Building GloveCom 2.0 gave us practical experience in Flutter mobile development, computer vision, hand tracking, gesture recognition, cloud integration, state management, and text-to-speech technologies.

We learned that an assistive technology solution should not only focus on recognition accuracy but also provide a simple and accessible user experience.

The project also helped us understand how computer vision and mobile technologies can be combined to solve real-world communication challenges without requiring specialized hardware.

Future Plans

Future versions of GloveCom 2.0 can include more advanced AI-based gesture recognition, support for a larger sign-language vocabulary, continuous sentence recognition, multilingual translation, improved personalization, and better recognition in challenging environments.

We also plan to improve the recognition model so that the application can understand more complex sequences of signs and provide more natural real-time communication.

Our goal is to develop GloveCom into an accessible AI-powered communication platform that helps reduce barriers between sign-language users and the wider community.

Built With

Share this project:

Updates

Submission history