Lmpool

preflight plan
durable monitor
Project and Model Pool
Input datasets
Visual job designer

Inspiration

I originally built LMPool when I wanted to study thousands of conversations which i had with coding agents over a full year: where they succeed, where alignment gaps emerge, and which conditions lead to better outcomes.

Doing that reliably and within reasonable time meant managing context limits, rate limits, cost, concurrency, invalid outputs, and multiple model deployments. I could not find one tool that handled those concerns as a coherent workflow, so I built it.

What it does

LMPool turns large LLM workloads, e.g. completions, annotations and judgements over large datasets into typed, declarative execution plans. Developers describe their jobs, inputs, model capacity, limits, and policies; LMPool validates the workload, estimates cost, plans concurrency, and executes it durably, while enabling monitoring.

LMPool Studio is a tool fully developed using Codex and Sol, which adds a visual workspace for designing jobs, uploading datasets, previewing plans, launching immutable revisions, and monitoring progress, retries, model assignments, lineage, and cost.

How I built it

I am the project's solo builder. Before Build Week, LMPool existed as a capacity-aware Python execution library. It was already mostly built with Codex using GPT and Sol models. During Build Week, I redesigned it around declarative jobs and durable runs, then built Studio as a thin visual control plane over the same core APIs.

I used Codex with GPT‑5.6 Sol at high reasoning effort as an engineering collaborator for architecture analysis, implementation, testing, debugging, and reviewing boundaries. I retained the product decisions and final judgment over every change.

Challenges we ran into

The hardest part was correctly recovering after a crash without losing accepted work or hiding an uncertain provider call is much harder.

Also had to make complex planning information understandable without turning Studio into a wall of infrastructure details. Finally, I wanted the demo to use real public data while remaining reproducible, safe, and honest about what was executed.

Accomplishments that I'm proud of

I am proud that LMPool now treats an LLM workload more like a query plan than a fragile loop of API calls.

During Build Week, I shipped the durable declarative lifecycle, visual Studio, immutable dataset and revision handling, cost and capacity previews, detailed run monitoring, recovery behavior, a provenance-pinned coding-agent dataset example, and deterministic browser and backend tests.

What we learned

I learned that reliability needs to be visible before execution, not added only after something fails. Developers need to understand what will run, where it will run, what it may cost, and how it can recover.

I also learned that AI collaboration works best when the human remains explicit about product boundaries and evidence. Codex accelerated the engineering enormously, but clear constraints and careful verification were still essential.

What's next for LMPool

I want LMPool to become a general execution layer for large-scale, multimodal research—not only text annotation.

That means supporting audio, images, and video as first-class inputs: evaluating thousands of coaching calls, finding patterns across video archives, labeling image collections, and studying large corpora of human– agent interactions.

I also want to support massive map-reduce research workflows. A researcher could analyze thousands of papers, codebases, support conversations, incident reports, or long recordings in parallel, then progressively reduce the results into an evidence-linked synthesis.

The goal is to let people ask questions that are too large for one prompt or one model call, while LMPool manages capacity, cost, validation, recovery, provenance, and reproducibility.