GestureGuard AI

web interface and hardware connections (light on and light off) are attached here
hardware connections (light off) are attached here

InspiratiInspiration

Communication and independent living can be challenging for speech-disabled individuals, especially during emergencies. Existing solutions often depend on voice commands, physical switches, or dedicated devices that may not always be accessible. We wanted to create an AI-powered, contactless system that allows users to communicate their needs, control appliances, and request help using simple hand gestures, thereby improving accessibility, safety, and independence.

What it does

GestureGuard AI is an intelligent gesture recognition and caregiver alert system that uses computer vision to detect predefined hand gestures in real time. The recognized gestures are translated into actions such as turning lights and fans on or off, while emergency gestures automatically trigger alerts to caregivers. The system operates through a web browser and communicates wirelessly with an ESP32-based IoT controller, enabling seamless and touch-free interaction.

How we built it

We developed the system using MediaPipe for hand landmark detection and TensorFlow.js for gesture classification. A custom gesture dataset was collected and used to train the AI model. The web-based interface processes live camera input, recognizes gestures, and sends commands to an ESP32 microcontroller through wireless communication. Relay modules and motor driver circuits control home appliances, while the Twilio API is integrated to send emergency notifications to caregivers when specific gestures are detected.

Challenges we ran into

One of the major challenges was achieving accurate gesture recognition under different lighting conditions and hand positions. Distinguishing between similar gestures while maintaining real-time performance was also difficult. Ensuring reliable communication between the browser, ESP32 controller, and notification system required careful testing and optimization. Additionally, reducing false detections for emergency gestures was essential to improve system reliability.

Accomplishments that we're proud of

We successfully built a real-time AI-powered gesture recognition system that can control appliances and generate emergency alerts without requiring speech or physical interaction. The browser-based design eliminates the need for dedicated mobile applications, making the solution more accessible and platform-independent. We are proud of creating a practical assistive technology that combines AI, IoT, and smart automation to support speech-disabled individuals.

What we learned

Through this project, we gained valuable experience in computer vision, machine learning, IoT integration, and real-time web application development. We learned how gesture datasets influence model performance, how to optimize AI models for browser environments, and how to integrate multiple technologies into a reliable assistive solution. We also developed a deeper understanding of accessibility-focused design and the importance of user-centered innovation.

What's next for GestureGuard AI

Our future plans include expanding the gesture vocabulary, supporting multiple languages and custom user-defined gestures, and improving recognition accuracy using advanced deep-learning models. We also aim to integrate additional smart-home devices, voice feedback, and cloud-based monitoring features. In the long term, GestureGuard AI can evolve into a comprehensive assistive platform that supports speech-disabled, elderly, and physically challenged individuals in achieving greater independence and safety.on