FOUND is largely inspired by Dina Wind—specifically, by her unique ability to see and unlock the true potential of found objects. In fact, many of the game’s primary functions are to encourage players to see and interact with the environment and objects around them.

From there, we launched into an exploration about search and discovery. How could we teach museum goers a new way to discover and see the art on display?

Team & Player Inspiration The “Ah-ha!” moment was sparked by a Drexel university design co-op named Radnyee Joshi, who offered up that the game—which, at the time, played more like digital iSpy—could function similarly to Pac Man, only the museum guests could be both the ghosts and Mrs. Pac Man.

Creative Every element of FOUND is meant to be a reflection of the museum experience, drawing inspiration from visitor pins, maps, frames and museum objects like the Ghost sculpture in the Grand Stairwell.

Game Play To inject an element of interactivity and, ultimately, an AR/mixed reality component, we drew inspiration from contemporary hits like Pokemon Go!.

Put simply, we describe FOUND like this: iSpy meets Pac Man, meets Indiana Jones.

What it does

FOUND encourages museum goers to explore the nooks and crannies of museum by motivating players through a series of scavenger hunts. The game leverages simple, highly recognizable features to make art discovery more accessible to many types of guests.

Players choose from two teams: Ghosts or Runners, which determines the style of gameplay they will experience.

Runners Runners are given a list of artifacts to “capture” from the museum. To capture the items, Runners explore the museum gallery by gallery. Once they have entered a gallery containing a desired artifact, the game will notify the Runner and encourage them to seek out the artifact (which is contained within a museum object).

Once the Runner believes they have found the artifact, they take a photo of the object containing the artifact. If they have truly found the correct artifact and object, they’ll earn points. If they are incorrect, they’ll lose the artifact and have to come back for it later.

The more artifacts a Runner captures, the more points they earn.

Ghosts Ghosts earn points by “protecting” priceless artifacts in the museum. Ghosts begin the game simply by exploring the museum at their leisure. They do not receive a list of artifact to capture.

Similarly to Runners, Ghosts will be alerted any time they enter a gallery which contains an artifact. However, the Ghost cannot earn points without a Runner present.

If a Runner is present (or enters soon after) in the same gallery, the Ghost will have the option to “Challenge” the Runner, forcing the Runner to locate and capture the artifact in just 30 seconds. Should the Runner fail, the Ghost succeeds in protecting the artifact and earns points.

Points & Games A new game is started each time a player visits the museum. The game is completed once a player either quits or restarts the game, or once a Runner has captured all of the artifacts on his or her list.

If a Player achieves Leader Board status, they will be encouraged to mark their achievement by using 3 initials.

How we built it

FOUND uses locative data from the PMA API and from iBeacons to determine a player's proximity to a gallery object. Once the player has captured an item, the application employs an artificial intelligence verification process to verify the correct object has been "captured."

Game experience and story were concepted and developed the ol' fashion way: on a whiteboard, and thereafter digitized using Adobe Experience Design (XD). The application is programmed via React Native to function seamlessly between iOS and Android, and is powered by the grit and sheer willpower of our developer team.

Challenges we ran into

Creative & UX Challenges

  • Developing a game experience that was accessible to a broad audience of players
  • Creating a sense of urgency without interrupting other museum guests
  • Giving players a reason to interact with each other in real life
  • Keeping the art safe

Tech & Development Challenges

  • Figuring out to use the beacons to accurately tell where in the museum a user is located because latitude and longitude coordinates aren't granular enough.
  • Figuring out what gallery the user is in based on the beacon(s) they are near, figuring what artwork is near that beacon
  • One of the most challenging points was figuring how to verify that what the user took a picture of the is the art object we told them to take a picture of.
  • The only challenge bigger than how to program the algorithm that compares the user taken art object image to the art object image provided to us by PMA was figuring out how to use React Native for iOS development. We wanted to use it to drastically reduce development time and make complex tasks easier but as with everything, there was a trade off. Even though React Native does provide quite a few benefits, it took quite a few days to get up, running and accustomed to the way to do things in React Native.

Accomplishments that we're proud of

Creative & UX Accomplishments

  • App is still educational in nature
  • The game forces museum guests to take a second look at the art on display
  • Gameplay is fun and easy-to-use!
  • User interface is simple and intuitive

API & Beacons At first, we built a wrapper around PMA's API that provided information like what gallery a user is based on beacon location, what artwork is nearby them and added in some things specific to our game like a poll to see whether or not a runner is being challenged and a poll to see whether or not (for ghosts) their are any runners nearby that they can challenge.

(Since then PMA has added to their API what gallery a beacon is in and the artwork in the gallery a beacon is in, but since we built our own wrapper we decided to stick with it.)

AI Verification One of our biggest challenges was to verify that players were “capturing” the objects we had instructed them to.

We used Artificial Intelligence to verify the objects in both the provided image of art object we told them to take a picture of and comparing those results to the the objects in the art object they just took a picture of

We took into account that AI doesn't always return 100% accurate information and for that reason we programmed our image matching algorithm to tell the user they have successfully taken a picture of the art object we told them to take a picture of if 40% of the objects in the art object the user took a picture of are in the art object provided to us by PMA. Previously, we had it at 60% but though the testing we conducted at the museum we found that a 60% match was to rigorous.

React Native Our development team learned an entirely new programming language/framework in a week.

What we learned

  • We learned how to have one code base for that builds an app for two platforms (iOS and Android).
  • We learned how to quickly and efficiently build a pretty full featured app that access native device functionality like the camera and the GPS chip, without using the programming language that is usually used to develop apps for that platform.
  • We also learned more about team accountability and how to work better as an agile team. As agency workers, we are accustomed to having a project management to support us.

What's next for Found

Adventure Play The prototype application currently isolates play to a specific exhibit—Asian Art. The next iteration will be developed with an Adventure-style play in mind that crosses exhibits and floors, encouraging more and more exploration of the museum.

Team Forums While FOUND is an inherently social game that encourages teams to seek one another out for help or conquest, we’re hoping that future iterations would include a Waze-like team communication alert and traditional message board.

Items In addition to the artifacts players already compete to find, future iterations of FOUND will allow players can also pick up helpful times along the way. The game will not notify players about them, but will pick them up for you if you are in the exact right spot. Objects can often be found in less commonly visited nooks and crannies of the museum.

Built With

  • ai
  • ai-verification
  • beacons
  • grit-and-determination
  • react-native
  • xd
Share this project: