Leap Motion is great, but sometimes the user can notice latency which might ruin the amazing experience. So we wrote a program that will expect the movement to make up for the latency.
What it does
It learns the user's behavior to make predictions of the user's next move for the next couple frames
How we built it
We used Unity3D to work with the Leap, and collected the hand's movement data while playing "rock paper scissors." Then we exported the location of the palm, thumb, and index as an input for our neural net to guess the next positions.
Challenges we ran into
Training the neural net to finish on time and making the net time-efficient enough to make predictions in less than 6 frames.
Accomplishments that we're proud of
We can make graphs of the expected movement for the next couple of frames :)
What we learned
How to use Leap Motion w/ Unity, and how to export the data to compute w/ neural networks
What's next for Leap Motion Action Prediction
We will make it work simultaneously with Unity in real-time to enhance the user's experience with Leap Motion P.S. Undefeatable "rock paper scissors" champion