nano-swe

Inspiration

Inspired by Andrej Karpathy's nanochat — a full LLM training pipeline small enough to read end-to-end. We wanted the same for agentic coding RL: a nano-scale, hackable version of the SWE-bench pipeline instead of a heavyweight framework.

What it does

Turns SWE-Gym / SWE-bench Verified into a working coding-agent RL pipeline: converts raw instances into self-contained "Harbor" tasks, runs an OpenHands agent per task in a real Daytona sandbox, grades it against the real test suite, reports pass@k/avg@k to W&B for eval, and bridges that same harness into the RL trainer as the rollout policy.

How we built it

Dataset conversion → eval harness (OpenHands + Daytona + Fireworks, validated against a real instance by hand) → switched to prebuilt per-instance Docker images for speed → built a chat proxy so OpenHands can drive the trainer's token-in/token-out vLLM policy while it silently tracks action-vs-observation token spans and logprobs → YAML recipes on top of the trainer's CLI.

Challenges we ran into

• The forked trainer code imported from a sibling package that didn't exist in this repo.
• Docker image tags must be lowercase; one dataset's instance IDs aren't.
• pip install -e . builds in an isolated env, so pinning an old setuptools outside it did nothing.
• No token-level plumbing exists between a chat-based agent and a token-in/token-out RL trainer — had to build the bridge ourselves.

Accomplishments that we're proud of

Every layer was verified against real data before trusting it: a hand-run patch application to confirm reward 0/1 logic, a live Docker Hub check to confirm prebuilt images exist, unit tests for the token/action-span bookkeeping before it ever touched a GPU.

What we learned

Reuse an existing minimal framework's primitives rather than reinventing FSDP/vLLM/Ray — the leverage is in the glue (data format, sandboxing, the agent↔trainer bridge), not the backend.

What's next for nano-swe

Build the prompt/SFT dataset loaders the trainer still needs, pre-build Docker images per (repo, version) for SWE-Gym training (not just eval), and run a first real RL training job.

Built With

• daytona
• fireworks
• fsdp
• ray
• vllm