Problem: It's a HUGE HASSLE to keep rewinding a video (using your hands) when learning. e.g. learning songs on the guitar, dance moves, cooking, etc. :(
The Dream: There should be a hands-free way to rewind videos, someone save us!
What it does
Achieves the dream of using your voice to navigate around a video such as going forward, backward, starting, pausing, yay! ^_^
How I built it
- Google Assistant (Integration with Google Homes or Google Assistant on your phone)
- DialogFlow (Framework to handle speech intent, slot filling, fulfilment, and all the conversational flow)
- Cloud Functions (Fulfilment endpoint for DialogFlow to push to)
- Pubsub (Event-bus to handling streaming commands at scale)
- Cloud Functions are publishers
- Mobile Clients are subscribers
- Flutter (Framework for creating apps on iOS and Android phones and tablets)
- Text-to-Speech (text processor for input from microphone)
- Google Cloud (publisher for Google Pubsub)
- Youtube API (video player on mobile app)
- ❤️ from Nick!
Challenges I ran into
- Terrible Latency for Google Assistant to Pubsub (over 4 seconds for a simple word)
- Unacceptable for real world use case UX wise :( ...
- No streaming text-to-speech interfaces for mobile
- Google Home Mini struggling to connect to nwhacks WiFi network T_T