Your earbuds are the controller. Astra is the opponent.

Growing up, games gave us an excuse to actually be together.

Twister, Monopoly, and especially the Wii turned living rooms into something everyone could be part of. You did not need to be a gamer. Someone handed you a Wii Remote, you swung it around, probably hit someone standing too close, and suddenly everyone wanted a turn.

We feel like we have lost a little bit of that.

Now everyone has a ridiculously powerful computer in their pocket, but most of the time we are using it alone. Scrolling, watching, replying, moving on. Everyone is caught up in their own feed and their own life.

So for Hack the North, we wanted to build something that did the opposite.

Something a group of people could gather around. Something slightly ridiculous. Something you could hand to a stranger and have them understand in five seconds.

So we turned an AirPod into a Wii-style motion controller and built an arcade around it.

Pick up an earbud. Tilt it to navigate. Swing it like a tennis racket. Challenge an AI opponent, slice through Neon Rush, or connect an iPhone and duel your friend.

It started as a project about bringing some of that physical, social fun back.

Then, because we are hackers and apparently could not leave it there, we started wondering:

What happens when the other person holding the controller is an AI?

How it works

The arcade is built natively with Swift, SwiftUI, SceneKit, and Core Motion.

When you calibrate the AirPod, its current orientation becomes your neutral position. From there, rotating the earbud rotates the racket in-game.

Swinging randomly is not enough either. Your racket still has to physically intersect with the ball for the hit to count.

Our How it works screen exposes the live motion readings, grip mapping, and racket output, so you can actually see an AirPod becoming a controller in real time.

Because headphone motion gives us rotation rather than absolute hand position, optional camera tracking can provide a separate position signal. The public headphone API also only gives us one active AirPods motion stream at a time, so our two-player mode pairs an AirPod with an iPhone.

Astra, the AI opponent

Once tennis worked, a scripted opponent felt a little boring.

In Astra · OpenAI API mode, we send the current game state and five legal returns to GPT-6 Astra through the OpenAI Responses API.

The model has to choose one of those shots. Its decision changes the target, timing, flight duration, and stroke of the next return while our native game engine handles the physics, movement, animations, and actual racket contact.

Everything runs asynchronously, so the game keeps moving while Astra thinks.

We also built an AI shot decisions panel where you can see what state the model received, what it chose, what shot actually happened, and how long the request took.

We did not want the AI to feel like magic hidden behind the game. Part of the fun is being able to watch what it is actually deciding.

Then we let Codex play

At some point we had the obvious terrible idea:

What if the AI that helped us build the game could actually play it too?

So we gave Codex the visible game window.

Instead of choosing a shot through an API, Codex can actually observe the interface, move the far-side player, and press Rival Swing, while a human stands on the other side swinging an AirPod.

Computer use is naturally slower than our direct API integration, so we slow the incoming play and buffer early swings for these experiments.

It is a little janky. It is also extremely fun to watch.

And that is kind of the point.

The development assistant became a character inside the thing it helped us build.

Building it with Codex

Codex was part of the project throughout the weekend, helping us build and test our Astra integration, provider architecture, sensor views, decision tracking, and regression tests.

It also helped catch bugs that were very much our fault.

Our first AI comparison, for example, reported seven instant rate-limit failures. Digging into the results showed that none of those requests had even reached the providers. Our own scheduler was sometimes waking fractional timers too early and rejecting them locally.

We fixed the timing logic, separated local failures from provider failures, and added a regression test designed specifically to break it again.

Computer-use testing also found a menu freeze caused by configuration reads happening during UI rendering. We moved them off the UI thread, added a timeout, and kept the interface responsive.

Codex did not just help us write the game. It helped us break it.

Which, at a hackathon, might be even more useful.

A small AI experiment

We also ran a controlled comparison between Astra and Jev using the same 20 synthetic game states, five legal shots, tactical objective, request cadence, and six-second deadline.

Each state ran twice per provider.

In our corrected live run, both returned 40 valid decisions from 40 requests.

Median request time was:

  • Astra: 1,836.5 ms
  • Jev: 322 ms

This is not a ranking of which model is better at tennis. It simply measures decision delivery through the API setups we used.

Why we built it

At the end of the weekend, the part we care about most is not the benchmark.

It is watching someone walk over, get handed an AirPod with absolutely no context, swing it once, see the racket move, and immediately understand the game.

Hackathons are one of the few places where you can spend a weekend asking questions like "Can an AirPod be a tennis racket?" and have that be considered productive.

We wanted to build something playful, physical, social, and a little unnecessary.

Something that gets people off their feeds for a minute and gives them a reason to play together again.

If someone picks up the earbud, smiles, and asks for another round, we built what we came here to build.

Built With

  • baseten
  • codex
  • core-motion
  • gpt-6-astra
  • openai
  • solana
  • swift
  • vercel
  • vision
Share this project:

Updates

Submission history