Beat-the-fly

can you beat a fly in crossy road?

Comment

Inspiration

MaleCNS, the first full synapse-level map of a fly brain, went public not long before HackGT. Flies are unreasonably good at not getting swatted, and that's not vague instinct, it's wiring you can actually download now. I wanted to see if that wiring could sit in a browser well enough that a person could feel it playing against them in real time, instead of it just being a diagram in a connectomics paper. So I built a game where the opponent isn't a scripted AI, it's a frozen model of an actual dissected fly's brain.

What it does

Beat-the-fly is a split screen Crossy Road. You hop across traffic on the left, a fly does the same on the right, both on the same seeded map. You get 30 seconds, whoever's made it further or is still alive wins.

The fly isn't scripted. Three regions of its actual brain, antennal lobe, mushroom body, and central complex, run as spiking networks trained on real MaleCNS wiring. Every 260ms they vote, blended toward central complex, and that becomes a prior for a look-ahead search that picks the actual hop to ensure a valid move is made. The sidebar shows what the connectome prefers versus what the fly did, plus offline proof numbers: intact wiring gets the right call 95.5% of the time, shuffled wiring drops under 10% - this suggest that we aren't cheating and a proper wired brain is crucial to success.

How we built it

Frontend is Vite, React, and TypeScript, rendering one Three.js canvas split into two viewports with a scissor test. Both halves share a seeded lane generator.

The brain starts outside the browser. A Python and PyTorch pipeline pulls the MaleCNS connectome as flat feather files, builds per-region graphs for AL, MB, and CX, and trains each as a LIF network with real synaptic weights frozen, only a thin encoder and decoder trainable. That gets reimplemented as a small TypeScript LIF engine running client side, with a selfcheck against the PyTorch output on load.

Match data goes through a Node server into Postgres via Tiger Data, deployed to a Vultr VPS through GitHub Actions.

Challenges we ran into

Two things ate most of my time. First, getting the brain atlas to highlight the right structures. Neuroglancer segments the fly brain by ID, and I had one wrong for a while, so the viewer was quietly lighting up the wrong region while the numbers underneath were still fine. I only caught it by checking segment by segment.

Second, harder to fix, was balancing how much freedom the connectome gets against how much the search constrains it. Too much freedom and the fly gets erratic, since central complex alone collapses toward one answer around half the time. Too much constraint and the search just takes over, and the sidebar stops matching what the fly does. That tradeoff lives in one number, and getting it to feel like the fly was actually thinking took a lot of back and forth.

Accomplishments that we're proud of

Getting a published connectome running as a spiking network in a browser, fast enough to drive a real-time game, and having a way to prove it isn't just for show, is what I'm proudest of. It would have been a lot easier to fake a convincing fly AI with a handful of if-statements. Doing it with a frozen model of a real nervous system, and being able to point at eval numbers instead of just asserting it works, is what made this feel like the actual project, with the game as the wrapper around it rather than the other way around.

What we learned

Real biological wiring doesn't behave like a hand-tuned network. Central complex collapsing to one output on its own was the clearest lesson, these regions aren't equally decisive alone, they need each other, and finding that balance was as much a design problem as an engineering one. I also learned there's a real difference between AI driving the character and AI biasing a search that drives the character.

What's next for Beat-the-fly

While researching this I ran into dopamine-based reinforcement learning in real fly brains, and that feels like the natural next step, letting the connectome adjust based on whether it's winning or losing instead of staying frozen after training. I'd also like to bring in more of the brain beyond AL, MB, and CX, more regions talking to each other, closer to something that actually behaves like a full nervous system rather than three regions voting in isolation. That's a much bigger project than a hackathon weekend, but it's the direction I want to keep pulling on.

Share this project:

Updates

Submission history