We wanted to create an interactive experience that lets users contribute to the visuals at an event. Panl is a cross-platform web app that lets anyone contribute from any device, and see themselves and their friends show up on the big screen.
What it does
Panl uses WebRTC to capture a 3 second animated selfie from which it is pushed to event's Panl in real time, leveraging Firebase. Brands can set up a Panl at an event and create user engagement but also capture demographic from the crowd using our real-time data analytics dashboard. The demographic data is sourced from a still frame from the user's selfie, from which IBM Watson image recognition pulls demographic data. The Panl design also changes dynamically based on this demographic.
How we built it
The selife camera is built with Node.js, Express.js and takes advantage of WebRTC to enable the user's camera. This web app is hosted on Heroku and uses Firebase to store assets.
The asset ID is then sent to a Python, Flask app (hosted on App Engine) which fetches the assets in real time, extracts a still frame, runs Watson's facial recognition API to extract demographic data, both of which are then persisted in Firebase.
The aggregate demographic data is then updated and reflected in real time on the data visualization dashboard, running on the Flask app.
The Panl projected on the event projector also runs on the Flask app and is updated on real-time based on the assets stored in Firebase.
Challenges we ran into
Neither of us knew Node.js beforehand, it was a steep learning curve. Firebase was also a relatively new skill. We ran into issues with SSL encryption. Extracting frames from a GIF had no good libraries, so we had to modify existing code snippet which was producing artifacts in the images. Eventually we patched it to meet our standards.
Accomplishments that we're proud of
Building two fully functioning apps between two developers in 24 hours!
What we learned
What's next for Panl
Use at different events.