iSpyAI

Inspiration

Take another look at the world of augmented reality games. Games like Ingress and Pokemon Go aim to bring players into the real world, but these often result in players simply standing around and staring at their screens.

Technology works best when it gets out of the way. We wanted to build a game that allows children to discover and explore the real world, with the technology assisting the gameplay instead of taking over.

Having been given access to Microsoft Azure's powerful Vision API, we decided to harness its potency to reinvent the way kids interact with the world, and we built iSpyAI.

iSpyAI is a game built for the real world, to be played in the real world.

What it does

Players join a game session created by a host, and they are tasked to find objects in their immediate surroundings. This enables children to interact with the people and the world around them in order to find the object. When they find the object, they just have to point their phones at the object and tap a button. The app will immediately return whether they have found the correct object, and points will be respectively awarded. There will also be a leaderboard for all players.

How we built it

The core of iSpyAI is the Android Application that runs on the devices of all the players. A real-time database of players, scores and game sessions is tracked with the help of Google Firebase.

When a user creates a game, a new SessionID is created in the Firebase database. Other players can then join the game using the session ID that is displayed on the host's phone. The managing of the multiple users, scores, and game session states are constantly synced between all devices in the game session using Firebase.

When the user finds an object and takes a picture to verify their find, the picture is sent from the app instance to Microsoft Azure's Vision API and the API returns the tags of the objects found. If the tags match the object required, a success message is displayed and points are awarded to the player. The leaderboard is instantly updated with the new score across all app instances on all participating devices in the same game session. In-game we have three screens showing various data, one for user metrics, such as username and leaderboards, one for the camera itself and for checking objectives.

Challenges we ran into

All of them. For starters, none of the team members had ever seen android studio or app-development before. We spent the first few hours, browsing for tutorials and scrolling through loads of documentation.

(Half the team might have slept twice as much as they should have)

Accomplishments that we're proud of

Staying alive. Working prototype. We are just glad you are reading this <3

What we learned

That app development is hard, and confusing And that we need better laptops to run android studio

What's next for iSpyAI

We believe that we can bring this game from a hyperlocalised setting to a larger scale. This would enable greater creativity in terms of objects to find and will spur real-world interaction between strangers.