Probe the Fly

A 3D game inspired from real fruit fly connectome, probe neural hotspots, watch behaviors emerge, and learn to control the fly through memory.

Comment

Inspiration

A complete connectome can show us every neuron and millions of synaptic connections, but it does not automatically explain how those connections create behavior. We wanted to turn this complex scientific dataset into something people could explore through play.

Probe the Fly asks a simple question: can you learn a brain well enough to control a fly?

Instead of using movement buttons, players interact directly with a 3D fruit-fly brain. They probe neural hotspots, observe the resulting behavior, and remember what each hotspot does to complete increasingly difficult tasks.

What it does

Probe the Fly is an interactive neuroscience and memory game built around neurons from the complete adult male fruit-fly central nervous system connectome.

Players can:

  1. Rotate and explore a 3D brain.
  2. Probe an unlabeled neural hotspot.
  3. Watch a pulse travel through a representative pathway of real reconstructed neurons.
  4. Observe the fly respond with a behavior such as turning or feeding.
  5. Record discoveries in a field notebook.
  6. Complete challenges by recalling the correct hotspots without relying on labels.
  7. Receive a recall report showing which neural hotspots they remember well and which ones they keep forgetting.
  8. Play adaptive weak-spot challenges that are built around their individual memory gaps.

The brain is the controller. There are no ordinary movement buttons.

Examples include left and right steering pathways based on DNa02 neurons and a representative feeding pathway beginning with AN13B002, also known as Dandelion.

How we built it

Game technology

The game runs entirely in the browser as a lightweight 3D web experience.

Our core game stack is:

  • three.js for rendering the 3D brain, neuron geometry, pulses, and interactive hotspots
  • Vite for the frontend build and development environment
  • Python for offline connectome processing, neuron queries, graph searches, and geometry preparation
  • Tiger Data for storing gameplay events and memory statistics using PostgreSQL + TimescaleDB
  • Python API for lightweight communication between the browser and the database
  • Browser-generated UUIDs + localStorage for anonymous player identity

The game is designed so that the core experience does not depend on the backend. If the API is unavailable, the player can still play normally.

Scientific data

We used the complete connectome of the adult male fruit fly, Drosophila melanogaster, containing approximately 166,700 neurons and 125 million synapses. The dataset was released by Google Research and HHMI Janelia and is available through neuPrint and Neuroglancer.

We used:

  • neuPrint to query neurons and directed synaptic connections
  • Neuroglancer to inspect reconstructed neurons and verify their anatomy
  • Published neuroscience research to associate candidate neuron types with behaviors

Pathway pipeline

We built a reusable Python pipeline that:

  1. Finds all neurons belonging to a literature-supported cell type.
  2. Separates left and right neuron instances when necessary.
  3. Retrieves their direct downstream connections.
  4. Tests multiple synapse-weight thresholds locally.
  5. Searches a bounded number of downstream layers.
  6. Flags possible motor or output endpoints.
  7. Ranks candidate paths using connection weights.
  8. Exports the selected pathway, evidence, and geometry metadata.

Biological endpoints are reviewed by a person instead of being accepted solely because they have the largest synapse count.

Pulse model

The connectome is a static wiring diagram, not a recording of neural activity. We therefore created a simplified and explainable pulse model.

For a path with connection weights (w_i), we score its strength using:

$$ \text{path score} = \sum_i \log(1 + w_i) $$

The model converts the selected path and connection strengths into visual pulse order, timing, and brightness for the game.

3D geometry

Neuron reconstructions are exported from neuPrint as SWC skeletons. A Blender Python script:

  • Reconstructs every node-to-parent connection
  • Builds thin tube geometry
  • Preserves all neurons in a shared coordinate system
  • Centers and scales the entire pathway together
  • Names each object using its Male CNS body ID
  • Exports the pathway as a GLB model for the game

The game then illuminates the neuron objects in pathway order before triggering the corresponding fly animation.

Challenges we ran into

Connectivity does not equal behavior

The connectome tells us which neurons connect, but not what activating each neuron will make the fly do. We had to combine connectome data with published behavioral research and clearly separate scientific evidence from our simplified model.

Different neuron classes need different searches

A descending neuron may connect directly to a motor neuron, while an ascending or sensory pathway can require several intermediate neurons. We redesigned our pipeline to support both direct pathways and bounded multi-hop searches.

The strongest connection is not always the correct target

Synapse weight measures anatomical connection strength, but biological relevance also depends on neuron type, anatomy, pathway direction, and published evidence. We added explicit human-review checkpoints instead of automatically choosing the highest-weight result.

Accomplishments that we're proud of

  • Built the experience around real neuron reconstructions and real synaptic connections rather than decorative brain graphics
  • Created a reusable pipeline from neuPrint queries to verified paths, GLB geometry, metadata, and game assets
  • Developed explainable graph-search and scoring methods instead of using an unsupported black box prediction
  • Connected neuroscience, mathematical modeling, 3D visualization, and game design in one project
  • Represented uncertainty honestly while still creating an approachable and playable experience

What we learned

We learned that a connectome is a wiring diagram, not a complete explanation of behavior. Scientific interpretation requires combining anatomy, connectivity, experiments, and literature.

We also learned that simplifying science for a game does not mean hiding uncertainty. Our system explicitly distinguishes between:

  • Real neuron shapes and connections
  • Literature-supported behavioral associations
  • Our simplified pulse-propagation model
  • Hand-authored mappings from motor output to game animation

Technically, we learned how to query a large connectome, perform bounded graph searches, preserve neuron coordinate systems, convert SWC skeletons into GLB assets, and coordinate different parts of a project through shared JSON schemas.

What's next for Probe the Fly

Next, we would like to:

  • Add more literature-verified behaviors and pathways
  • Expand bilateral and lesion-based challenges
  • Improve pulse timing using more detailed biological constraints
  • Add classroom annotations explaining each neuron and brain region
  • Optimize the neuron geometry for lower-powered devices
  • Let players compare alternative pathways that produce similar behaviors
  • Build additional levels focused on scientific reasoning, memory, and neural redundancy

Scientific transparency

Neuron shapes and connections in Probe the Fly come from the adult male fruit-fly CNS connectome. Behavioral associations come from published research, not from the connectome alone. Pulse propagation is our simplified model rather than a validated simulation of a living fly, and the final mapping from motor output to game animation is hand-authored.

Game architecture

The project is divided into an offline data pipeline, browser game, and lightweight backend.

OFFLINE
Male CNS connectome
        ↓
Python / neuPrint
        ↓
Select neural hotspots
        ↓
Graph/pathway search
        ↓
Pulse propagation model
        ↓
Neuron geometry
        ↓
hotspots.json + neurons.glb

GAME
three.js + Vite
        ↓
Load precomputed data
        ↓
Click hotspot
        ↓
Pulse animation
        ↓
Fly behavior
        ↓
Notebook entry
        ↓
Recall + scoring

BACKEND
Probe event
        ↓
Python API
        ↓
Tiger Data
        ↓
Memory statistics
        ↓
Weak-spot challenges

Built With

Share this project:

Updates