Liveboards - The Future of Message Boards

Inspiration

We had (what we thought were) a bunch of cool ideas for a mixed AR experience. Over the holiday break we sat down with nine high school and college kids who we believe are a little more in touch with "what's next" than we are. research team

They were very specific in their guidance, and thought it was hilarious to call me "Boomer". insight

The more we talked, the more we zoomed in on the judging criteria: Tech Implementation: we were going to need to hit outside APIs, build image processing pipelines in addition to navigating Zappar + three.js

Design: we started with customer input and then did about six revs on the overall flow.

Potential Impact: our research panel was very clear about "is valuable first, cool looking second". That didn't stop us from adding a floating Solo cup for the fraternity party flyer or a VW van for the rideshare.

Quality of the Idea: again, we started with potential end users to develop some loose product "goal posts".

What it does

Liveboards takes the mess that are college and coffeehouse message boards and turns them into the dynamic connections we demand in 2023. We build a series of formats -- party, rideshare, for sale, football tickets available, movie night -- and created the "live" experience that turns a one way piece of paper into the connection it could be.

For example, the QR code on the "bike for sale" ad brings up an experience with better images of the bike and a button for the logical next step -- texting the owner to meet up.

Football tickets include an image of the view of the field from the seats and an "add to calendar". It also includes the idea of "These Tickets Are No Longer Available".

On top of that, with the help of ChatGPT, we've made all of the flyers multi-lingual...the flyer for the Swedish Clubs Vaffeldagen has a second code for Swedish.

How we built it

We chose ZapAR Universal for three-js. On the backend an AWS Python Lambda does some data management.

We used the following platforms, asset libraries and tools to create our AR experience:

Universal AR SDK for Three.js
AWS: Python Lambda | API Gateway | RDS
Blender for image development
A few public domain 3D images
The SeatGeek API to programmatically generate the seat image to include in the "Tickets for Sale" AR experience
3rd party ICS (calendar) event creator to dynamically build "click to add to calendar"
3rd party QR code generator to build QR code sets in multiple languages.
ChatGPT for translation
Lots 'o Stack Overflow

Challenges we ran into

We had one giant 300 lines of code beatdown The experience flow is that the user scans a flyer on the board and is brought to an enhanced AR experience that lubricates the logical digital next steps of the flyer. It's clunky to have the user then return to the camera to scan the next code on a flyer....so, as part of our programmatic QR code development for the flyers, we were also training the images to generate the Zappar ".zpt" images. The idea was you could dismiss the AR flyer and then the tracker could find another code on a flyer and you could click-and-go without leaving the Zappar experience.

There were no good pure JS examples of how to load and track and be able to click on 25 images, but we grinded it out and even added a little flair of the overlay image reflecting the overall total clicks on that link (a loose proxy for popularity). Then, the beatdown...our QR codes were only marginally different than each other (different enough for a QR code reader) and so whenever it "found" one, it often thought it found 3 or 4 and the screen was a mess. We were experimenting with some workarounds, but ran out of clock.

Accomplishments that we're proud of

We went back to our original research panel (and to be clear, that isn't their real picture above) and they approved v1 of the Liveboards (with a healthy set of "notes"). Despite the beatdown, as a developer it was very viscerally satisfying to figure out how to load, scene-ify and make clickable 30 QR codes in pursuit of a great user experience.

What we learned

We were pretty new to AR/VR and so we came a long way up the virtual learning curve. Zappar has laid out the tracking and placement and scene management in a wonderfully organized way...and once we figured out that all of the variables being created were readable objects, a lot of the management became much easier.

We also learned from our research panel that the pure novelty of AR is no longer enough to sustain a project -- they took a pretty sharp eye to "did you make it better, or just different?". Kids in their late teens can be very rough.

What's next for Liveboards - The Future of Message Boards

We're committed to solving that multi-tracking brutality and putting out a v2 with a better user experience. We still have to hunt for a business model if we're going to continue to put any effort behind a v3.