Daby

• 

Inspiration

Daby was inspired by a simple question: what would an AI research assistant look like if it could design, run, and audit experiments involving living-neuron models?

Cortical Labs’ work suggests a future where biological neurons participate directly in computation. But experimenting with these systems is slow, delicate, and scientifically risky. We wanted to shorten that loop without allowing an AI model to invent evidence or overstate what the neurons learned.

What it does

Daby is an AI-assisted experimentation system for a simulated cortical culture learning a Pong-like task.

A researcher defines safe parameter ranges, and Daby:

• Runs a baseline BL1 neural simulation.
• Gives the measured baseline—not fabricated context—to Fireworks.
• Uses Fireworks to select exactly three distinct parameter variations.
• Runs each candidate in an isolated Daytona sandbox.
• Measures behavioral improvement and neural stability deterministically.
• Rejects candidates with unstable or hyperactive neural activity.
• Ranks the remaining experiments using recorded evidence.
• Uses Braintrust to audit whether the AI followed the research contract.

Daby does not claim to prove biological learning or predict live CL1 performance. It reports whether a specific BL1 run exhibited a bounded Pong-learning signal while remaining neurally stable.

How we built it

The experimental core is a JAX-based BL1 simulation containing 100 Izhikevich neurons, online STDP, stimulation encoding, spike decoding, and a Pong environment.

Fireworks acts as the experiment-selection agent. It receives the baseline measurements, the researcher-approved ranges, and a strict structured-output contract. It returns three candidates that vary:

• Minimum position-encoding frequency.
• Maximum position-encoding frequency.
• Neural decoding-window duration.

Reward frequency and duration remain null because the current BL1 runner does not support changing them.

Daytona executes the baseline and candidate runs inside isolated sandboxes. Each run produces an artifact containing its configuration, game events, rally lengths, population firing rates, and provenance.

The final ranking is deliberately not produced by the language model. Deterministic analysis calculates behavioral improvement, neural stability, and the winning candidate.

Braintrust traces the Fireworks decision. It records sanitized inputs, model metadata, latency, token usage, validated outputs, and whether the selection contract passed—without logging credentials.

Challenges we ran into

• Our first Fireworks requests failed before inference because billing was suspended and the requested models were unavailable. Once billing was resolved, we identified gpt-oss-20b as a working model and completed a real structured selection request.

• Fireworks RFT was more complicated than simply uploading examples. It required a prompt-only dataset, a deterministic reward evaluator, and an evaluator package compatible with the training environment. We went through multiple packaging and wrapper iterations while testing that contract.

• The model that worked for inference, gpt-oss-20b, was reported as not supporting fine-tuning. That forced us to evaluate smaller tunable alternatives rather than assuming the production inference model could also be used for RFT.

• SFT introduced a different data problem. Although several saved selection outputs existed, only one contained the complete baseline context needed for a scientifically grounded training example. We chose not to duplicate that example or invent missing labels merely to satisfy a dataset-size requirement.

• We prepared validators, dataset exporters, and deterministic RFT scoring logic, but did not represent training as complete or launch a paid job without a validated dataset.

• Neural activity sometimes became hyperactive. Some runs appeared behaviorally strong while being scientifically unstable, so Daby needed a hard stability gate rather than simply selecting the highest score.

• Daytona sandboxes were intentionally short-lived and deleted after artifact collection to avoid unnecessary resource usage. This preserved the results but made the completed environments disappear from the live dashboard, creating a demo-visibility challenge.

• Coordinating Fireworks, Daytona, BL1, Braintrust, and local analysis required careful artifact contracts. Partial provider failures could not be allowed to become fake successful results.

• Braintrust tracing had to remain optional and fail-open so an observability outage could never prevent the underlying experiment from completing. We also had to resolve project and organization configuration before producing a usable trace link.

• Presenting the workflow clearly was difficult. We needed to make a complex scientific system understandable without hiding uncertainty or implying that a simulation result was proof of learning in living neurons.

Accomplishments that we're proud of

• We completed a real baseline and three isolated candidate experiments.
• Fireworks produced one genuine, schema-validated selection of three bounded variations.
• The system rejected unstable neural behavior before ranking candidates.
• Every scientific conclusion can be traced to saved measurements.
• Braintrust captured a real, passing audit of the AI selection decision.
• Daytona environments were cleaned up safely while their artifacts and identifiers were preserved.
• Unsupported parameters were explicitly labeled instead of silently simulated.
• We maintained a strict distinction between a BL1 learning signal and evidence about live CL1 hardware.
• When the available fine-tuning data was insufficient, we stopped rather than manufacturing training examples.

What we learned

AI is useful for proposing experiments, but it should not be the final authority on whether an experiment succeeded.

The strongest architecture separated responsibilities:

• Fireworks proposes.
• Daytona executes.
• BL1 produces neural and behavioral evidence.
• Deterministic code evaluates and ranks.
• Braintrust audits the AI’s reasoning contract.

We also learned that fine-tuning infrastructure is only one part of RFT. The more important work is defining a reward function that captures scientific validity, assembling genuinely grounded examples, and ensuring the chosen model is actually tunable.

Most importantly, research software needs to preserve uncertainty. A transparent incomplete result is more valuable than a polished but unsupported claim.

What’s next for Daby

• Collect more genuinely grounded baseline-and-selection examples.
• Complete Fireworks RFT or SFT using a verified tunable model.
• Compare the tuned selector against the base model on held-out experiments.
• Use Braintrust to evaluate contract compliance and evidence grounding across those comparisons.
• Run more seeds and repeated BL1 trials to strengthen statistical confidence.
• Improve persistent Daytona run evidence without leaving paid resources active.
• Expand the supported BL1 parameter surface while preserving researcher-defined safety limits.
• Eventually test the workflow with carefully supervised CL1 hardware—but only after the simulation evidence and safety controls are substantially stronger.

video

https://www.loom.com/share/e792dcc128ec4e11a7a63fbe89f493f6

Built With

• braintrust
• daytona
• fireworks