Inspiration
GestureAssistant - Unleash the power of your hand and automate anything
What it does
This is a simple prototype for GestureAssistant - a Desktop Personal Virtual Assistant which understands your gestures(so it understands you:) ) and turns them into powerful commands which will automate your Desktop without you pressing a key..(but you will have to start the assistant yourself:))
It can automate almost anything that a computer can do..so it can do everything..
Scenario 1 : Lets say, your mom doesn't know how to operate a computer [I am sure there are lots of moms], and you are tired teaching them the same thing again and again.
I am in the same situation. And I applied my Engineering Brain to solve this problem.
Scenario 2 : Suppose you are getting bored. You want to watch a movie on youtube or listen to a song or send an email or play a game or chat with friends on facebook or peep into instagram or tweet on twitter or upload this code on Github:) or this or that and so on.............. Infinite scenarios... But you dont want to go to your Start Menu --> Open Chrome --> enter the URL and wait for it to load..
One Solution...Gestures
Simply give a gesture command and get things done quickly...
How I built it
Its built in python. I have used Pytorch to recognize the hand gestures and then associated them with commands...
Currently, it can do the following stuff : 1) Open Gmail 2) Start video player 3) Start Microsoft Word 4) Open Google Play Music 5) Open Devpost.com
See github code for more details.
Challenges I ran into
Started with deep learning for the first time. It was difficult to understand initially..but loved it after completion
Accomplishments that I'm proud of
Built the project in 4 days ( and nights) !!! Improved in 4 months...
What I learned
Lot of things : 1)How to use Pytorch with python 2) what is deep learning 3) How to automate things on Windows 4) How to build an app in 4 days
What's next for GestureAssistant - Your hand based Virtual Assistant
I would learn image processing using torchvision in depth to more accuretely analyze the gestures and add support for more gestures.
Log in or sign up for Devpost to join the conversation.