• 

  Many Mini Labs logo

Many Mini Labs

Small Macs. Many experiments. Better models.

Turn spare Apple-silicon Macs into a distributed AI frontier lab for parallel GPT-designed model-architecture experiments—no GPU cluster needed.

How it works:

GPT-5.6 proposes bounded architectures, distributed local MLX workers run independent trials, and a trusted coordinator compares their safe checkpoints and asks what the lab should test next.

Judge instructions

Clone the repository on any machine with Node 20+ and make, then run:

make demo

Open the printed localhost URL.

On an Apple-silicon Mac, make quickstart runs genuine lightweight MLX training. make verify runs the deterministic judge suite.

Judges who configure their own OpenAI key can optionally run make live-demo; it requests fresh GPT-5.6 designs before training and a fresh evidence-driven successor after the trusted development winner is frozen.

Optionally, judges with two Apple-silicon Macs on one trusted LAN can reproduce the distributed worker path with MML_PHYSICAL_MULTI_MAC_ATTESTED=true make lan-demo on Mac A. It prints a complete ready-to-paste Mac B command, a separate one-use pairing code, and a complete private-IP fallback command. The launcher journals both worker paths and exports an integrity-checked evidence bundle automatically; this path also needs no OpenAI key.

Inspiration

AI model architecture research needs many controlled experiments, yet useful experimentation is concentrated in expensive centralized compute environments. At the same time, small teams, researchers and students already own capable Apple-silicon Macs that sit idle for long periods.

Independent architecture trials are naturally parallel. The missing piece is more than a job queue; it is an agent-driven research loop that can form useful hypotheses, preserve scientific controls, compare results honestly, and decide what to test next.

What it does

GPT-5.6 proposes three materially different causal-transformer architectures inside a strict grammar. Each proposal must state a hypothesis, expected tradeoff, and falsifying result. The coordinating server validates every proposal, adds a fixed baseline, and dispatches the four independent trials across paired Apple-silicon Macs so the architecture experiments can run in parallel.

Each worker trains locally with native MLX. Every comparable trial uses the same pinned repository-authored corpus revision, deterministic byte tokenizer, splits, optimizer policy, predicted-token budget, final evaluation policy, and seed policy. Microbatch size can adapt to memory; effective batch and total data exposure cannot.

Workers upload safe tensor checkpoints—not executable code. A trusted coordinator evaluator reconstructs each validated architecture and recomputes development loss. Worker-reported loss, timing, device, and memory remain telemetry. The dashboard keeps development loss, estimated compute, parameter count, duration, memory, and evaluator latency separate, then shows the frozen selection without hiding tradeoffs in a composite score.

After selection, the same GPT path interprets the four trusted results and proposes one follow-up architecture for the lab to test next. The strongest recorded run contains validated live GPT initial and successor responses; the proposed successor is not presented as a completed trial. The default judge demo replays this sanitized, integrity-checked evidence without making a new API call, so judges need no credentials or network access. Only after the winner is frozen does the coordinator reveal the sealed holdout once.

Why it is different

BOINC, volunteer-compute systems, and schedulers distribute work. NAS and HPO systems search configuration spaces. Many Mini Labs does not claim to reinvent those components.

Its product contribution is the integrated loop: a distributed AI frontier lab for parallel GPT-designed model-architecture experiments on ordinary Apple-silicon Macs, with controlled budgets, coordinator-owned checkpoint evaluation, evidence-driven successor design, and a commit-pinned, integrity-checked replay. In live mode GPT-5.6 is the lab designer, not a chatbot attached to a scheduler; deterministic code still owns validation, evaluation, and selection. The prototype needs no centralized GPU cluster.

How GPT-5.6 is used

The product uses the OpenAI Responses API with explicit model ID gpt-5.6-sol, maximum reasoning effort (ultra), strict Structured Outputs, and store:false. The output schema contains architecture data and research explanations only—never code. Local validation enforces all ranges, head divisibility, exact fields, unique IDs, and material differences. One bounded structured repair is allowed; otherwise the app falls back to a clearly labeled validated fixture.

GPT-5.6 cannot change data, optimizer, token budget, evaluator, selection policy, checkpoint pointer, or holdout state. Deterministic code retains scientific authority.

The live evidence bundle records outcome: live-valid, response IDs, resolved model, schema names, prompt hashes, and designSource: gpt-5.6-live for the initial design and successor. Fixture runs remain labeled separately.

How it was built with Codex

One primary Codex task built Many Mini Labs end to end. Its required Session ID is 019f8412-fcbf-77e0-8961-bc09d1039d25. Codex translated the product idea into a fixed scientific protocol, built the TypeScript coordinator/contracts/dashboard and Python/MLX worker, exercised native training and browser paths, maintained the claims ledger, and prepared the judge and submission assets.

Three focused specialist reviews attacked MLX numerics, coordinator security/crash behavior, and judge clarity. High-impact findings became boundaries or tests: streamed content-addressed uploads, evaluator isolation, telemetry-independent selection, fixture/physical evidence labels, and a two-minute replay path.

Challenges

• Keeping training budgets comparable while letting different Macs choose safe microbatches.
• Treating uploaded checkpoints as untrusted data without pretending a same-user subprocess is a security sandbox.
• Making lost upload responses idempotent without accepting a changed checkpoint.
• Separating worker telemetry from evaluation authority throughout state, UI, and evidence.
• Giving GPT meaningful research agency without allowing it to emit code or rewrite scientific controls.
• Making a complete distributed story judgeable without requiring judges to own Apple silicon or wait for training.

Accomplishments

• A bounded generic MLX causal-transformer implementation with byte tokenizer, gradient accumulation, safe tensor checkpoints, and trusted reload/evaluation.
• Expiring one-use pairing, authenticated workers, idempotent leases/uploads, hash-linked JSONL state, atomic snapshots, and restart recovery.
• Strict GPT-5.6 architecture and successor contracts with local validation and safe fixture fallback.
• One live/replay dashboard that makes the distributed lab loop visible: GPT hypotheses, worker assignments, an equal-budget trial ledger, the trusted decision, the proposed next test, holdout state, evidence integrity, and limitations.
• make demo, make quickstart, and make verify judge paths.
• A two-command trusted-LAN path whose owner-attested run produced coordinator-journaled, integrity-checked evidence from two physical Apple-silicon Macs.
• A sanitized replay whose evidence class prevents fixture or process-level workers from becoming a physical-device claim.

Evidence statement

GPT-5.6 Sol returned three locally validated candidate architectures. On one M4 / 24 GB Apple-silicon Mac, pinned MLX 0.32.0 trained the fixed baseline and all three candidates under the same 8,192-token bundled-corpus budget. Trusted development NLL was 3.733717 for shallow-wide GEGLU, 3.780392 for the baseline, 3.906117 for deep-narrow RMS, and 4.173807 for lean-FF post-norm. The frozen rule selected shallow-wide GEGLU; its one-time holdout NLL was 3.753893. GPT-5.6 then used those four trusted results to propose the untrained follow-up “Mid-depth GEGLU bridge.”

Separately, an owner-attested trusted-LAN run paired two physical Apple-silicon Macs as distinct observed workers. The coordinator journal records different trial assignments, equal 8,192-token budgets, native MLX/Metal telemetry, distinct safe checkpoints, and coordinator-owned evaluations: the balanced baseline measured 3.780392 development NLL (8080dd96…770e40a) and the deeper gated learner measured 3.905744 (24cedb19…c344053). The integrity-checked physical-multi-mac bundle is direct prototype evidence that the distributed two-Mac worker path ran end to end.

The physical LAN proof used offline validated-fixture designs and completed two of four trials; its recorded training windows did not overlap. The complete live-GPT experiment above remains single-Mac, and neither run represents the full 1,048,576-token budget. The default judge demo is a read-only replay of these recorded evidence bundles, not a fresh run.

What we learned

The strongest boundary is simple: a worker can supply compute and a checkpoint, but it cannot supply truth. Central evaluation, fixed data exposure, explicit evidence classes, and visible negative results matter more than a clever score. GPT becomes more useful when asked for a falsifiable experiment and a successor from evidence—not when asked to generate arbitrary training code.

What is next

The immediate engineering steps are clear: refuse new work until the research plan is durably recorded, fail readiness if a recorded checkpoint is missing, make every protocol promise executable rather than descriptive, and extend evidence checks from “unchanged export” to “internally consistent experiment.” After that come redundant evaluation, signed evidence, stronger local isolation, more seeds, broader Apple-silicon measurement, community-owned labs, private datasets, and richer experiment grammars. A marketplace, arbitrary workload execution, and frontier-scale training are not implied.

Limits

Small-model prototype; Apple silicon only; trusted participants; local HTTP; one coordinator; no accounts, payments, arbitrary jobs, adversarial-worker defense, remote attestation, cryptographic proof of computation, Neural Engine claim, frontier-scale training, or OpenAI endorsement. Exact limitations are canonical in docs/LIMITATIONS.md.

Built With

• chatgpt-5.6-sol
• codex
• github
• vscode