In the words of Richard Stallman, "hacking means exploring the limits of what is possible, in a spirit of playful cleverness".

With that spirit in mind, we set out to explore the limits of a device called the Leap motion. It's a small infrared sensor that's designed to track hand movement. We wanted to use its 3d depth sensing capabilities to build the 21st century equivalent of the classic pin toy, repackaged as a collaborative art project. Two recycled cardboard kiosks house our Leap motion sensors, which are connected to laptops running custom software that lets users interact with the Leap sensors, take pictures, and upload them to our online gallery

How it works

IMPRINT has three primary components: the kiosk application, server backend, and web frontend.

Kiosk app

We wrote our kiosk application in a hybrid of Python and HTML/Javascript. It uses the Leap API to construct a streaming image from the data coming from the 3d sensor, then pipes that to a browser application with our UI. When the user clicks the shutter button, the image is uploaded to Imgur's CDN and the url is passed to our web backend.

Web backend

Our web backend is an Express/MongoDB/Node application running on Heroku. It exposes a JSON API as well as a WebSocket interface to enable live updates.

Web frontend

Our web gallery is a ReactJS application that talks to the backend via WebSockets, enabling seamless injection of new gallery images without page reloads, as well as live-updating timestamps.

Challenges we ran into

At around midnight, our team member who was working on the Python part of our tech stack had a heroic battle with Python Version Conflict Hell while developing the video streaming code that powers our app. After help from mentors and a thorough review of Google, we were able to get past it and get video streaming up and running.

Accomplishments that we're proud of

We're happy that we managed to do something very unconventional & interesting with a device that's really meant for a completely different purpose (finger/hand tracking).

What we learned

  1. Be very careful what version of Python you link against if you have multiple versions installed.
  2. Uncheck ALL of the "auto" boxes when you're working with the Leap motion API.
  3. Just because you CAN stream video using x/mixed-replace headers doesn't mean you SHOULD.

What's next for IMPRINT

We'd really like to get IMPRINT to support generating animated GIFs.

Share this project: