AI Virtual Piano

• 

  Play piano via keyboard, auto songs, voice ("Play B4"), or gestures—touchless experience shown in demo.

Inspiration
We wanted to create a musical experience that is easy to use, touch-free, and powered by AI. Inspired by gesture control and voice-assist technologies, our goal was to rethink how a piano could be played. There would be no physical keys, just your hands and voice.

What it does
AI Virtual Piano detects hand gestures using a webcam and translates them into piano key presses. It also supports keyboard typing and voice commands, like "Play B4." Users can see recent notes, hear sounds, and even play a full song, such as "Happy Birthday."

How we built it
We used:

• Python for scripting
  
• OpenCV and MediaPipe for real-time hand tracking
  
• PyAutoGUI for simulating keypresses
  
• SpeechRecognition and Pyttsx3 for voice input and output
  
• Tkinter for the GUI with a visual keyboard
  

Challenges we faced

• Ensuring proper hand tracking in different lighting
  
• Syncing gesture input with real-time sound playback
  
• Integrating voice input without lag
  
• Smoothly handling various inputs (gesture, voice, keyboard)
  

Accomplishments we're proud of

• Built a fully touchless, interactive piano system
  
• Successfully integrated gesture and voice control
  
• Played actual melodies using only hand gestures
  
• Created a clean GUI with real-time feedback
  

What we learned

• Fine-tuning hand landmark data for specific tasks
  
• Combining multiple AI interfaces in one workflow
  
• Designing for usability and responsiveness in real-time apps
  

What's next for AI Virtual Piano

• Add multi-hand chord recognition
  
• Improve the voice assistant to support song requests
  
• Deploy as a web-based app using Flask
  
• Integrate MIDI support and custom sound banks
  
• Enable recording and playback features for users
  

Built With

• gui
• mediapipe
• numpy
• opencv
• pyautogui
• pygame
• python
• pyttsx3
• speechrecognition
• tkinter
• vs