Lithops

A Gemini agent fleet that runs a company: it writes executable models of the business, tests them against reality, and acts. CEO-Bench is our benchmark. Write your own adapter, point it at yours, run.

Comment

Inspiration

Over the last three years we went from plain LLM calls with structured output that worked at random (a prayer in the prompt: "please return valid JSON") to recursive self-evolving agent frameworks full of magic incantations. Our expectations grew with the hype: now the agent is supposed to run a company.

In June, Princeton published CEO-Bench: run a simulated SaaS startup for 504 days, partially observable market, delayed and coupled consequences, a competitor that never sleeps. The authors call the measured capability "steering intelligence", and most frontier models go bankrupt before day 500. My bet was: Gemini 3.7 Flash can run a company with an optimized harness. To test this, I built Lithops.

What it does

Lithops is a harness for operating a company over a long horizon; during the hackathon it ran against CEO-Bench. Every simulated week it normalizes observations, scores the forecasts it committed before acting, updates a world model, simulates candidate strategies in a Monte Carlo engine, and lets Gemini 3.7 Flash pick one bounded plan from priced evaluation cards (every candidate plan arrives with a simulated dollar outcome attached). The core thesis: let Gemini write its own predictive code, forecast what each concrete week will bring, and build its world model out of the accumulated history of those forecasts.

How we built it

Lithops has three parts: a hand-written Python kernel, the Gemini agents running through the Google ADK, and the world-model code those agents write. The kernel owns everything that has to be trustworthy. Every simulated week it normalizes the raw observations through a unit registry, so a monthly metric never silently mixes with a weekly one and missing data stays missing instead of becoming zero. Before the model acts, the kernel writes the chosen plan's forecasts into a ledger, and when the real outcome arrives a week later, it scores them. It prices every candidate plan by simulating it in 1,000 Monte Carlo worlds and prints the result on an evaluation card: expected outcome, named risks, and one hard rule, which is that a plan able to bankrupt the company never reaches the model at all. Whatever the model picks, an action gateway executes through schema-validated tools with idempotency keys and receipts, and the week advances.

Gemini 3.7 Flash cannot modify any of that code. It plays two roles: an executive that reads the observation, the cards and its own forecast-versus-actual history, then returns one structured weekly plan for about $0.006 a decision; and five model authors (acquisition, pricing, retention, plus two deliberately competing theories of conversion) that write the world models themselves, small Python modules predicting what each lever will do. A freshly written module does not touch a single decision. It first runs in a sandbox with no network and no secrets, then gets backtested on the time-ordered history, and only if it beats the current champion does it get promoted. The original hand-written simulator stays in the registry as the baseline, so every challenger always has an opponent.

Cloud Build builds the images, Secret Manager holds the keys, Cloud Logging carries a structured audit trail of every week, and a Cloud Run service serves the replay. During the hackathon week I ran 55 benchmark runs, 897 simulated weeks, about 17 years of company time, for $204 of inference. Then I rebuilt the image and launched one clean run from week zero on the final code. That run is the project's actual result, and its replay is live, ready to watch.

Challenges we ran into

  • Three consecutive runs generated 3,098 leads and converted zero, because the action layer and the observation layer silently disagreed about reality: ad spend executed with receipts and never appeared in observations. A drift instrument (the gauge that watches whether the market still behaves the way the fitted models assume) returned 0.0 when it could not parse data, which reads exactly like "no drift". World models fitted on a history with zero conversions predicted zero forever.
  • The executive never proposed an R&D program in 55 weeks, because a never-executed lever has no observations, so its payoff rested on a prior, so the forecast underpriced it, so it lost every comparison, so it was never executed. At the bottom of that chain sat a bug in the Monte Carlo rollouts (the simulations that price each candidate plan) valuing one $167k decision at minus $2.79M.
  • In the last clean run Gemini built the best early game any run on this seed has ever seen. Then a forecast said reverting its quality program beats adopting it by $88k, and it obeyed. Development spend stayed at zero for the remaining 48 weeks while a visible counter of competitor releases climbed from 1 to 32, and the company died on the same cliff a third time. Meanwhile its own world model, in code it wrote itself, had converged on the correct theory: conversion collapses when quality fails to cross a threshold that competitor releases keep raising. The knowledge was in the registry (the versioned store of all world-model code); the decision never reached for it.

Accomplishments that we're proud of

Raw Gemini Flash goes bankrupt on this benchmark three times out of three by day 150, and its best surviving run keeps $75k of the starting million. The same model inside Lithops never went bankrupt in any run of the project, kept $551-632k, and posted the seed's record early game (28 customers by week 12), for five to forty dollars of inference per run. A control run with a stronger model on identical instruments built growing revenue in twelve weeks, which means the instruments are sufficient and the model is the variable: the cleanest attribution this project produced.

The architecture itself is the accomplishment I trust most. Fifty-five runs and three ugly root-cause bugs later, the core loop never needed a rewrite: every fix landed in an instrument and an adapter.

What we learned

Eleven of twelve times, the harness was lying, and an instrument must be able to say "I don't know" as a value distinct from zero, because a silent 0.0 steers a year of decisions. Asking an LLM to imagine consequences is asking it to be a simulator, and it is measurably bad at that; asking it to write the simulator, then executing that code deterministically, works. And a cheap model with a correct theory, an exact price list and $700k in the bank still would not spend $167k to jump a bar it had itself measured. Appetite for irreversible commitment is the frontier, and per my own literature map it is one of the least-studied directions in long-horizon agent research.

What's next for Lithops

The next step is to keep tightening the harness until Gemini 3.7 Flash finishes a full 504-day CEO-Bench run in profit. Beyond that, the harness is the product: pointing Lithops at a different environment, including real business data behind the same veto-and-inform contract (deterministic code can block or flag a decision, but the model makes it), means writing just a new adapter.

Built With

Share this project:

Updates

Submission history