crackARjack

• 

  Launch App

• 

  Detect Dodger Stadium

• 

  Players identified, placed, and one tapped for info.

• 

  Merchandise recognized for in-app purchase

Inspiration

Looking up player info distracts you from the game. There's also no way to find out info about the park without leaving your seat, such as how long are the bathroom lines, or where are the vendors. And, why is it so hard to buy team merchandise that you see others wearing?

What it does

crackARjack is an augmented reality app that enhances the visitor experience at the ballpark. You can view the players on the field within the augmented reality view and tap them to learn details about the player. The app can also recognize merchandise and identify it so you can make a purchase in-app and have it delivered to you.

How I built it

I built an iOS app using ARKit and SceneKit to render the augmented reality view. I pull team and player info from MLB.com, parse it into info the app can use, fetch images for all the players from there as well, and generate an image view to display in the augmented reality scene. The images are placed based on player position. When you tap a player image, the app displays details about the player.

The app recognizes both 2D and 3D views. The ballpark is identified by recognizing its 2D picture of the field. That kicks off the fetch of the team and player info. Merchandise are recognized by their 3D image. I scanned 1 product (a hat) and the app recognizes it in the real world to allow you to further interact (such as getting details or making a purchase).

Challenges I ran into

This was a lot of code and processes to get working in 6 hours, and I was a team of one. Getting the MLB data took a bit of research, and I was limited to what I could find in the short time available. Rendering an AR scene properly when you're not in the real location was a big challenge. I printed out a picture of the Dodger Stadium field and used it as a mock real-world scene, but everything had to be adapted to the smaller scale, and flat surface.

Accomplishments that I'm proud of

Getting all of this code working in such a short time. The scene recognition and player positioning and interaction worked better than I expected.

What I learned

There's a LOT of baseball data and it's scattered in a wide variety of places. I also learned a lot about ARKit and scene composition.

What's next for crackARjack

Adding more venue content - images of all parks, for example. Getting more player info from appropriate sources. Adding real-time data for the game. Adding real merchandise for a park, and connecting to a real fulfillment backend.

Built With

• arkit
• ios
• mlb-apis
• scenekit
• swift