M.I.N.K.I

🌟 Minki – Bridging Silence with Technology

🧠 Inspiration

In a world full of communication, millions of deaf and mute individuals face daily challenges expressing themselves, especially in public places. We wanted to build something affordable, portable, and powerful—a solution that could serve as a real-time interpreter for sign language, making everyday interactions seamless and inclusive.

💡 What it does

Minki is a small, smart device that uses a camera module, Raspberry Pi, display screen, and audio output to interpret American Sign Language (ASL) gestures in real time and convert them into spoken and visual words.

Key Features:

🖐️ Real-time hand gesture recognition using MediaPipe and a custom ASL-trained ML model
📷 Captures signs via the camera and sends coordinates to the ML model
🔊 Converts recognized gestures into voice output using text-to-speech
🖥️ Simultaneously displays the interpreted text on the device screen
🔌 Fully portable and designed for public interaction (banks, stores, hospitals, etc.)

🛠️ How we built it

Hardware: Raspberry Pi 4, Pi Camera Module, Speaker, Touchscreen Display
Software:
- MediaPipe for gesture landmark detection
- Custom TensorFlow/Keras model trained on ASL dataset
- Python for backend logic
- Pyttsx3 for text-to-speech conversion
- Lightweight Flask server for modular testing
Deployment: The full model runs on the Raspberry Pi without needing internet—making it fast and offline-friendly.

🧩 Challenges we ran into

Achieving high FPS with limited Pi processing power
Gesture overlap (e.g., letters with similar signs)
Power management and device compactness
Fine-tuning the model to handle different hand sizes, lighting conditions, and backgrounds

🚀 Accomplishments that we're proud of

Successfully interpreting ASL gestures into speech with 85%+ accuracy
Fully self-contained device—no cloud dependency
Super intuitive UI and minimal latency
Designed with accessibility and cost-efficiency in mind

📚 What we learned

How to deploy ML models efficiently on edge devices
The intricacies of real-time gesture detection
Importance of optimizing UX for accessibility
Working with hardware-software integration

🔮 What’s next

🔁 Add support for two-way communication (speech to sign via an animated hand model)
🌍 Multi-language and regional sign language support
🤝 Partnering with NGOs and accessibility groups
📱 Companion mobile app for updates and customization