Dry Run

Describe your business. Tavily finds the news that hits it, Nemotron proposes plans, and a simulator tests every plan in several futures. You get a decision memo with sources.

Comment

Inspiration

A small business owner reads "coffee prices hit a record" and has to decide by Monday: raise prices, stock up, switch supplier, or wait? Big companies have analysts and scenario models for this; a café owner has a hunch. We wanted to give them a dry run: test the plan before betting the shop on it.

What it does

  1. Describe your business in plain words (or paste a shop link, or upload a price list photo).
  2. Tavily finds the news that hits it: supply news, plus local-market searches in your country's language (new rivals, demand shifts).
  3. Nemotron reads it: every number it takes from an article must come with a quote found word for word on the page.
  4. A Nemotron Ultra analyst proposes plans and picks the futures to test them in: shock levels, a rival's price move, a new player. It can even ask Tavily its own question mid-analysis.
  5. A deterministic simulator tests every plan in every future against "Do nothing", with 1,000 paths per cell and common random numbers. A pixel town of shoppers shows it happening.
  6. You get a decision memo: the plan, why, the bad case, the cash it ties up, its sources, and an explicit list of what isn't modelled.

The rule that makes it trustworthy: Nemotron makes the decisions (plans, shopper reactions, shock numbers). The NumPy engine computes every outcome. The model never does arithmetic.

We also backtest it honestly: past cases with Tavily date windows. The app lists every case, misses included (currently right in 7 of the 9 listed).

How we built it

  • Models (all via Nebius Token Factory, one OpenAI-compatible client): Nemotron 3 Ultra (analyst, thinking on), Nemotron 3 Super (profile and news parser), Nemotron 3.5 Lightning (shopper groups), Nemotron 3 Nano (shopper fallback), Nemotron 3.5 Super-VL (reads photos/PDFs).
  • Tavily: Search (news and local market, country, language, date windows for backtests), Extract (with query reranking, for quote verification), and Map (for pasted shop links). Every response is cached.
  • Backend: Python, FastAPI with Server-Sent Events, Pydantic v2 strict schemas, and a pure, seeded, unit-tested NumPy/SciPy engine.
  • Frontend: Next.js, Tailwind, Zustand, HTML5 Canvas pixel arena.
  • Hosting: the app runs on Render's free tier so the live demo stays up and free until judging ends. Public visitors get a recorded replay by default; live mode is rate-limited and has a daily spend cap.

Shoppers choose with a multinomial logit that includes an outside option (they can walk away):

$$ P_{ij} = \frac{e^{U_{ij}}}{e^{U_{i0}} + e^{U_{iA}} + e^{U_{iB}}} $$

$$ U_{ij} = \beta_{0j} - \alpha_i P_j + \gamma \ln(1+M_j) + \delta S_{ij} + L_{ij} $$

Here P is price, M is marketing, S is the shopper's sentiment, L is loyalty, and option 0 is walking away.

Lightning supplies the sentiment term; the engine supplies everything else.

Every LLM call uses strict json_schema with Pydantic validation and value clamping. It retries once, then uses a deterministic fallback, with per-tier timeouts. A per-model rate limiter reads the x-ratelimit-* headers. Every telemetry number in the UI is computed from real call logs, never hardcoded.

Challenges we ran into

  • maxLength cuts strings mid-word: strict-schema strings came back cut ("…within your $8,0"), so we size prompts to the limits.
  • Ultra ignores reasoning_budget: we control reasoning with prompt wording and max_tokens. Once, reasoning ate all 6,000 tokens and cut off the JSON.
  • Lightning shoppers can be inconsistent: they reacted harder to a 5% price rise than an 8% one, and scored standing loyalty instead of the reaction. Our backtests showed this drove the dollar claims, so we cut the sentiment weight from 0.8 to 0.05 and re-recorded every case.
  • News search returns noise: Instagram posts, sitemaps, an article about diamonds for a coffee query. We filter non-articles and require verbatim quotes, and an article whose parsed shock is all zero is skipped.
  • Model knowledge cutoff leaks into backtests: Ultra already knew the 2024 cocoa spike. We say so wherever backtest verdicts appear.
  • A tight budget: $25 of Token Factory credit and Tavily's free plan meant caching every response, capping live runs, and being careful about what we spent on testing.

Accomplishments that we're proud of

  • A model that never does arithmetic. Every dollar in the memo comes from a seeded, unit-tested engine, so any number can be traced and reproduced.
  • Grounding you can check. Every figure taken from the news carries a verbatim quote, verified against the page with Tavily Extract.
  • Honest backtests. We publish the misses next to the hits (7 of 9), and we cut our own most flattering knob when the data said it was inflating results.
  • Five Nemotron models in one pipeline, each in the tier it fits: Ultra thinks, Super parses, Lightning and Nano play shoppers, Super-VL reads photos.
  • Useful to a real owner. The memo says what the plan is, how bad the bad case gets, how much cash it ties up, and what we didn't model.
  • A live demo that stays up and free through judging, with a pixel town that makes the simulation easy to follow.

What we learned

  • LLMs are great decision-makers and bad calculators. Splitting decisions from arithmetic made the output both explainable and testable.
  • Honest evaluation hurts. Our first backtest numbers looked great until held-out cases showed the shopper sentiment was inflating them.
  • Grounding has to be checked, not trusted. A real quote can still be about the wrong product.
  • Strict schemas plus a deterministic fallback make agent pipelines far more reliable than clever prompting alone.

What's next for Dry Run

  • Run the 10,000-path Monte Carlo as a Nebius Serverless Job. The job code is written; we haven't deployed it, so we don't claim it.
  • Calibrate shopper reactions against real sales data.
  • Add more held-out backtests and a recurring "watch my business" news alert.

How to try it

Open https://dryrun-1hny.onrender.com and pick an example. It replays a recorded run for free. Press "Check live news" to make a real Tavily call. The free host may take about 50 s to wake up. To run it locally, see the README.

Built With

  • fastapi
  • game-theory
  • html5
  • monte-carlo
  • nashpy
  • nebius-serverless
  • nebius-token-factory
  • nemotron-3-super
  • nemotron-3-ultra
  • nemotron-3.5-lightning
  • next.js
  • numpy
  • nvidia-nemotron
  • openai-sdk
  • pydantic
  • python
  • react
  • scipy
  • server-sent-events
  • tailwindcss
  • tavily
  • typescript
  • zustand
Share this project:

Updates

Submission history