Sparsh Mukthi

Embracing Touchless Technology

Comment

Sparsh Mukthi

Inspiration

In today’s world, constant physical interaction with devices—keyboards, mice, touchscreens—creates inefficiencies and hygiene concerns, especially in sensitive environments such as hospitals, VR classrooms, and shared office spaces.
This led to the creation of Sparsh Mukthi, a system enabling touchless control through hand gestures and voice commands.
The vision is to build a hygienic, accessible, and futuristic interface that reduces touch dependency and expands inclusivity for differently-abled users.

What it does

Sparsh Mukthi enables users to control their system without physical contact by:

  • Recognizing voice commands to perform common actions (opening apps, sending messages, online search).
  • Detecting hand gestures to navigate screens, scroll, or interact with applications.
  • Providing context-aware automation so commands adapt to the active screen (e.g., WhatsApp vs. Chrome).

This combination of gesture and speech enables seamless human–computer interaction.

How we built it

  • Frontend: Minimalistic Python interface for executing voice and gesture commands.
  • Voice Recognition: Integrated speech-to-text APIs for accurate, fast command interpretation.
  • Gesture Recognition: Used computer vision techniques (OpenCV/MediaPipe) to convert hand movements into commands.
  • Context Awareness: Logic designed to adapt voice and gesture inputs depending on the active application window.
  • AI Layer: Enabled natural language understanding to avoid rigid or predefined command formats.

Challenges we ran into

Noise Sensitivity in Voice Recognition

Background noise interfered with processing voice commands.
Solution: Applied noise-cancellation filters and adjusted detection thresholds.

Gesture Accuracy in Low Light

Low lighting reduced hand-tracking stability.
Solution: Added preprocessing filters and fallback command options.

Context Switching

Understanding different app environments (like WhatsApp vs. browser) was complex.
Solution: Developed a modular context-handling engine for dynamic action interpretation.

Accomplishments that we're proud of

  • Developed a functional prototype combining voice and gesture recognition.
  • Achieved adaptability across multiple application environments.
  • Built with accessibility in mind to support differently-abled users.
  • Delivered a touchless, hygienic interaction method suitable for healthcare and education.

What we learned

  • The significance of human-centered design in AI-driven systems.
Share this project:

Updates