Leap Motion is great, but sometimes the user can notice latency which might ruin the amazing experience. So we wrote a program that will expect the movement to make up for the latency.

What it does

It learns the user's behavior to make predictions of the user's next move for the next couple frames

How we built it

We used Unity3D to work with the Leap, and collected the hand's movement data while playing "rock paper scissors." Then we exported the location of the palm, thumb, and index as an input for our neural net to guess the next positions.

Challenges we ran into

Training the neural net to finish on time and making the net time-efficient enough to make predictions in less than 6 frames.

Accomplishments that we're proud of

We can make graphs of the expected movement for the next couple of frames :)

What we learned

How to use Leap Motion w/ Unity, and how to export the data to compute w/ neural networks

What's next for Leap Motion Action Prediction

We will make it work simultaneously with Unity in real-time to enhance the user's experience with Leap Motion P.S. Undefeatable "rock paper scissors" champion

Share this project: