Frankenfly

We downloaded a real fruit fly's brain, wired it to a robot, and it chases you. Nobody wrote the code that makes it do that.

Inspiration

In June, Janelia and Cambridge released the complete wiring diagram of a male fruit fly's nervous system, and in September they published the paper on it. 166,700 neurons, over 124 million connections, free to download.

People went at it immediately. There are now sixty-odd public projects running that wiring: Doom, Mario 64, Minecraft, chess, Pong, blackjack, tic-tac-toe, a desktop pet that walks around your screen, one that sorts your email, one that trades stocks. We went looking for the one that had put it in something physical and there isn't one. Every fly in that list lives in a browser, a game engine, or a physics simulator.

So we built that one. A fly's eye-to-leg circuit driving a real robot across a real room toward a real person, and 150 neurons of the same connectome running on the conference badge in your pocket.

It chases you. We didn't write that part. That's the whole project.

What it does

The camera finds a person. Where they are in the frame becomes an angle, the angle becomes a current, and that current goes into the same cells a fly sees you with: LC10a, which is what a male fly tracks a mate with, plus the looming detectors that fire when something big comes at its face.

This is a circuit subset, and we say which one: 15,000 of the fly's 166,700 neurons, everything on a path from eye to descending neuron. The wiring is fixed. Nothing is ever rewired or added.

Then 15,000 neurons and 2,334,959 connections run the way real neurons do, one millisecond at a time. Out the other end we read the descending neurons, the only wires a fly has between its brain and its body. The gap between the left and right steering cell is how hard the robot turns, because that's how a fly turns.

There's no line anywhere that says "person on the left, turn left."

With no training of any kind it turns toward a person in 75% of runs and reaches and touches them 97.7% of the time. If you walk away it follows, keeping you in frame 100% of the time. Camera to wheels takes 15 ms, and we'd budgeted 100.

We assumed we'd cheated

Our first reaction to it working was that we'd screwed up somewhere and the robot was just chasing the biggest blob in frame. So most of the weekend went into trying to break it. Three of those attempts are on hotkeys, and at the table we hand the keyboard to the judge, because a control you ran yourself is one you believe.

Rewire it at random. Keep every neuron, every connection, every strength and sign, and the exact number of wires into and out of each cell. The only thing that changes is which cell connects to which. The eye fires just as hard, 69.9 Hz, completely unbothered. The steering cells go from 158 Hz to zero and the robot sits there while you walk past it.

That's the one that convinced us. Same parts, same amount of wiring, different map, nothing happens.

Pull out the tracking cells. 275 of them out of 15,000, the ones a male fly chases a mate with. Turn-toward goes from 78% to zero. Nothing else in the eye reaches the steering cells at all.

Pull them out a quarter at a time. Four presses: 158, 137, 83, 40, 0 Hz. It fades rather than flipping off, and the robot keeps turning until three quarters of them are gone, which is what a crowd of cells does and a switch doesn't. Put them back and it returns to exactly 158.

And one we ran offline, on the part we had to guess. The dataset says whether each cell excites or inhibits, except for 127 of the 15,000 where it doesn't know, and we assume those are excitatory. So we flipped all 127 to inhibitory, the worst case for that assumption, and ran the whole thing again. Turn-toward drops from 73% to 66% and every lesion result above is unchanged. The circuit gets quieter and stays the same circuit.

It also fits on your badge

Everyone here is carrying an ESP32, so we got 150 real connectome neurons running on ours: the fly's escape reflex, the circuit that makes it jump when you swat at it, ticking at 100 Hz and checked against the full simulation. Swat the badge and six LEDs flash white before you can pull your hand back. Hit B and it cuts the wires out of the eye, so the eye keeps firing, nothing reaches the escape cell, and nothing flashes. Same argument as the robot, in your pocket.

How we built it

Neurons pulled from neuPrint by cell type, expanded one hop so the real in-between cells are actually there instead of us inventing a shortcut. Connection counts set the strengths and the predicted neurotransmitter decides whether a connection excites or inhibits. Standard leaky integrate-and-fire on top, with constants from Shiu et al. 2024.

The encoder has two front ends. Training computes the fly's visual field straight from ground truth, so hundreds of practice arenas run in parallel and we never render a pixel. The robot computes the identical thing from the camera, and the brain genuinely can't tell which world it's in.

We never train a separate network, and that was a hard rule from the first hour. The only thing allowed to change is the strength of connections that already exist in the animal, with the fly's own dopamine cells as the reward signal. Excitatory stays excitatory, inhibitory stays inhibitory, nothing new ever gets created. In the shorthand the fly-connectome community uses: fixed wiring, only synaptic gains change, and the steering behaviour is there before any of them do.

Challenges we ran into

The forward-walking neurons never fired. There are two cells the textbooks say make a fly walk forward, and no matter what we showed the eye they sat at exactly zero. That killed us for most of Saturday, because a robot that turns beautifully and never moves is not a demo.

So we wrote a tool that peels inhibition off a neuron one source at a time, and it turns out there are two completely different reasons a cell can be silent. One neuron gets 801 connections straight from the tracking cells, more than any other cell in the circuit, and still never fires, because the same eye signal is driving the cells that hold it shut. Lift two of those off and it jumps to 91 Hz. Which makes sense once you look it up, because that cell steers a fly away from things, and a fly chasing a mate has no business firing it.

The two forward-walking cells are the opposite. Strip every inhibitory input off them and they still sit at zero. Nothing is holding them down, the signal just never arrives. So we read forward speed off the whole descending population instead. That's our engineering, not the fly's, and we say so wherever it comes up.

We made a prediction and the data said no. There's a relay region carrying 80% of the traffic between eye and steering, and it's the pathway the courtship literature already describes, so it looks essential. We predicted cutting it would kill tracking. It doesn't: tracking survives at 61%, against 71% for deleting the same number of random cells. The wiring diagram told us where the traffic goes and was still the wrong answer about what matters.

Four overnight training runs came back flat. The mushroom body is where a fly actually learns and here it does nothing, because none of our senses reach it hard enough to fire it. We also tried a constant forward push to get the robot moving, which raised the contact rate and pinned the steering cells at 300 Hz. A circuit screaming at its ceiling didn't learn anything, so we threw it out.

Accomplishments that we're proud of

The rewiring control. Everything else follows from being able to run it in front of someone on one keypress and have it fail the way we said it would beforehand.

Also 257 passing tests, and scripts/reproduce.py --quick re-derives the eight headline numbers on your machine in 12 seconds.

What we learned

Most of this brain is doing nothing here. Throw away the weakest 95% of the connections, 2.2 million of them, and it still turns toward a person 72% of the time. Throw away the same number at random and it's dead long before that. The whole behaviour fits in 2,211 neurons, 0.4% of what we started with, and that tiny version fails in all the same ways when you lesion it.

What's ours and what's the fly's. The steering is the fly's, untrained. Training only ever bought us speed, never behaviour: time to reach a person went from 4.25 s to 3.23 s and the success rate didn't move. The visual encoding, the forward read-out and the search-around-when-nobody's-there behaviour are ours, and we'd rather draw that line than have a judge draw it for us.

What's next

Legs, obviously. The map we downloaded includes the fly's spinal cord, which means the real motor neurons for all six legs are sitting in a file on this laptop. Right now we take signals meant for legs and hand them to two wheels, which is a bit like wiring someone's motor cortex to a golf cart. Everything we'd need to do it properly is already downloaded.

Credits

  • Male CNS connectome: Berg et al., "Sexual dimorphism in the complete Drosophila male central nervous system connectome," Cell, September 2026. neuPrint male-cns:v1.0, from HHMI Janelia and the University of Cambridge.
  • LIF model and model-ready connectivity: Shiu et al., Nature 2024, philshiu/Drosophila_brain_model (MIT).
  • FlyWire v783: Dorkenwald et al., Nature 2024. NumPy formulation follows flypoke (MIT). Detection: Ultralytics YOLO11n. Viewer: Three.js r180 (MIT).

Built With

Share this project:

Updates

Submission history