Let's select an adventure and begin!
First I should check what's in my inventory...
Awesome! I don't even remember applying. Sounds fun though!
My first clue. Hmm, I think there's something off about a few of the words. I wonder if they're related...
Could this be what the clue was referring to? I should scan it.
Oooh it's working!
WHOA! There's a clue hidden on this painting. I wonder what this one's about.
This might be the best feature. Instead of looking at the paper map I got when I entered, I can scroll around this unzoomable digial one...
Based on the tagline, it's probably clear that National Treasure was a huge influence on this project. This classic masterpiece took the American history that we learned and revealed a intertwining story hidden behind its most famous artifacts.
Our goal was to emulate the experience of solving puzzles and discovering hidden pieces of history for visitors of the Philadelphia Museum of Art (PMA). The PMA has a beautiful collection of American artwork and we wanted to build a story around these pieces. We also wanted to create a story that could scale to multiple adventures throughout various galleries or even the entire museum!
When evaluating technologies, we felt that augmented reality (AR) was a perfect fit for this experience. This allows our users to be fully immersed in the game while still enjoying the wonderful collection of curated artwork around them.
“The identity of the original found objects give way to a new work of art.” – Dina Wind
Just like Dina took unassuming objects and transformed them into stunning sculptures, we challenged ourselves to breathe new stories and experiences into the exquisite pieces found in the Philadelphia Museum of Art.
What it does
After turning on the AR-T, you notice two pieces of paper in your digital inventory.
One is a acceptance letter into a secret society, called The Artists. You learn that throughout history members of this group have hidden secrets and clues in their works of art. Before you are accepted into this society, you must uncover their secrets by tracing clues back in time.
The other is your first clue.
In this game, you'll be given clues that lead you to paintings, where you'll find another clue that leads to...yep, you guessed it, another painting. The clues range from puzzling riddles to clever limericks. When you "solve" the clue and find the painting, you'll scan it using the AR-T. Your next clue might be written in invisible ink or layered behind the painting - all of which the AR-T is capable of processing. You win the game after scanning the final painting, revealing a secret of The Artists.
The story we crafted links a New World immigrant artist to his grandson-in-law painter, then to his student, and so on. It features prominent contemporaries who worked and lived in Philadelphia, Pennsylvania between the 1750's to the late 1800's.
DISCLAIMER: We've intentionally left out specific details regarding paintings and artists featured in the game. Since this is a puzzle game, we wanted to avoid spoilers as much as possible. ...Kinda makes you want to play the game, huh?
How we built it
We built AR-T using Unity and ARToolkit, since we had experience using both. We extracted unique features from each piece of artwork allowing the AR library to distinguish them. Then we designed the UI and workflows and imported them into our Unity project.
To weave our story, we researched paintings (aka. reading gallery labels and Wikipedia) that were available to us. An Elasticsearch index and Kibana dashboard made navigating the information a breeze. Our goal was to find artists that were linked historically, crafting a narrative and brainstorming clues that linked American artists working in Philadelphia.
We then devised clues that linked each painting together using different AR effects and iterated on them based on feedback from our friends. Finally we play-tested our game with random strangers in the museum, which gave us some great ideas on improving the game.
The rest is history (and magic).
Challenges we ran into
We had a hard time getting a cross-platform iBeacon library to work in Unity. Our original plan was to use the location data to keep users in the right area, as a clue (think radioactivity locator), and to dynamically load AR data based on the closest galleries. Though, in the end, we decided it wasn't critical.
Oh and time. We lost track of that really quickly.
Accomplishments that we're proud of
- A working game on both Android and iOS #couldnthavemadewithoutunity
- Crafting a narrative that connects six related American painters, many of whom worked in Philadelphia #uwishunuthem
- Finding three groups of PMA visitors willing to try out our über-beta game instead of looking at Claude Monet's paintings #moremonetmoreproblems
What we learned
Sooooo much about American art!
Fun fact we learned: Did you know Gilbert Stuart (who was almost featured in our game) painted The Athenaeum, an unfinished portrait of George Washington, in Philadelphia? So what, right? Wrong! It's the portrait portrayed on the US one dollar bill!
What's next for AR-T
Over the course of the project we brainstormed a ton of ideas, but didn't have time to implement them all. Here's some of our favorites:
- More adventures! We would love to make more adventures that features paintings in the Eurpoean section of the museum. We could also try out some interesting AR overlays in the modern galleries, where it would be more realistic.
- Different difficulties and time lengths. This goes along with the previous, in that a variety of adventure types would allow visitors to Choose Their Own Adventure™ based on their preferences and time constraints in the museum.
- Enable scanning for more paintings in the museum. We proved this was possible for every painting we had data for, but took it out for performance reasons. We have a few tricks up our sleeve to optimize this, though.
What's next for Team Cage
After the hackathon, Joe and Nick will continue to be BFFs. We'll be on the lookout for opportunities to share our creative vision, as long as we can shamelessly promote our favorite movies. This hackathon was a great experience for us to work together outside of work and network with the Philly tech scene, so I'm sure we'll try another one.
As for Nic Cage, I'm sure he's off making a least three timeless blockbusters. After all, he is the real National Treasure.
- How do you pronounce "AR-T"?
Either "R. T." (Pirate noise + golf ball stand)
- What does AR-T stand for?
While many believe it stands for "Augmented Reality Tablet", it was really just bad word play on "AR" and "art".
- Wait, is AR-T the name of the game, or the device, or the project?
Yes. All of the above.
- What do you guys do when your not hackathon-ing?
We're both software engineers!
- Instead of writing a fake FAQ, shouldn't you be spending more time on the game?