Mandat on Nemotron

Tell it what you want and how much you can spend. Mandat, running on NVIDIA Nemotron via Nebius Token Factory, finds, negotiates, books and pays, never a cent over budget.

Comment

Try it in 2 minutes (judges)

  1. Open https://mandat-nebius.onrender.com on a phone or a desktop browser. No sign-up.
  2. Tap Continue with PayPal and log in on the PayPal sandbox page with this test buyer account (sandbox only, no real money):
  3. Type an errand with a budget, for example: Barcelona and Valencia by train, 3 to 6 November, a bed every night and a paella lunch, 250 € max. Or attach a photo of something to fix, or tap the microphone and just talk.

Inspiration

Everyone has a list: a trip we never get round to planning, trainers we would rather throw away than get fixed, flowers we meant to send. AI agents can already do the research. What stops people from letting them finish the job is the money: either the agent cannot pay, or it gets a card with no real limit and no record of what it did.

We wanted an agent you could hand a budget to the way you would hand it to a trusted assistant, built entirely on open models and open infrastructure: here is what I want, here is what I can spend, tell me when it is done.

What it does

You write, say or show what you need ("Barcelona and Valencia by train, 3 to 6 November, a bed every night and a paella lunch, 250 euros max", a photo of worn trainers, or just your voice) and the most you will spend. Mandat:

  • asks only what it must (a few questions with tap answers), then works on its own;
  • searches real places and real products (Google Places, OpenStreetMap, Channel3), and looks at product photos before suggesting anything;
  • negotiates with merchants' AI agents after checking each agent's signed identity card;
  • sends one-tap booking requests to small merchants without any AI;
  • holds deposits with PayPal (sandbox) instead of charging them, captured only when the merchant confirms, voided if plans change, refunded if needed;
  • keeps every mission inside its budget, at the autonomy level you chose, with a daily limit and one switch that stops everything;
  • works from a photo (it reads what the item is and what is wrong, then finds a repair or the closest equivalents under your budget) and by live voice (it answers out loud while it works);
  • splits the bill to the cent with a PayPal invoice and a QR code per friend.

Everything lands in an Activity view (every payment like a bank statement, with receipts and a dashboard you can question) and an Agenda of every booking and delivery.

How we built it on NVIDIA Nemotron and Nebius Token Factory

  • The agent's brain is NVIDIA Nemotron 3 Super (120B, A12B), served by Nebius Token Factory. Mandat is a tool-calling loop with 31 tools (search, inspect, quote, negotiate, request, hold, void, refund, move a booking, buy, split, remember, plan). Token Factory's OpenAI-compatible API meant the same client works with a base URL and a model name: no SDK lock-in.
  • Nemotron 3 Ultra (550B, A55B) is the senior planner that reviews every plan before money moves ("a dinner needs a table before a cake is delivered there"), and the fallback when a call fails. We measured it before choosing it: on six plans with a known flaw (or none), three runs each, Ultra found 18 of 18 real problems and invented none; Super found 16 and invented one, for 1.5 s less.
  • Photos (worn trainers, a product) are read by Gemma 3 27B on the same Token Factory endpoint, five times, keeping the majority spelling of any brand printed on the item, so the agent never names a brand it did not read.
  • The model proposes, the code decides. Amounts, budget, limits and the books are computed in code; every payment is checked before PayPal is called; a fact sheet and an exact calendar are injected at every turn.
  • A ceiling on our own bill. Every call's cost is counted from its tokens; the public demo stops calling the models at a fixed budget and says so plainly, while missions and payments stay available. A full birthday mission costs about $0.20 on Token Factory.
  • Node.js and Express, an installable web app, PayPal Server SDK and Agent Toolkit (sandbox). The judging demo runs on Render.

Challenges we ran into

We measured before we trusted. Our bench (scripts/bench-provider.mjs) runs the same realistic mission end to end on each model, approves every payment and prints the books.

  • Dates. Nemotron sometimes computed "this Saturday" wrong, and the plan reviewer, which had no calendar, called a correct date a mismatch and pushed the agent to refund a dinner. Fix: an exact 15-day calendar (weekday + date) injected at every turn and given to the reviewer.
  • Paying in pieces. A cake deposit was paid as 27 + 21 + 6. Fix in code: one deposit per booking; a grown order is a top-up, never a second payment.
  • Saying what was not done. The model once announced a deposit "released" that was not. Fix: the rules now say only what the tools confirmed, and the books shown to the user always come from code.
  • Tool arguments. One model sent the plan as a JSON string and another invented a phone number. Fix: tolerant parsing of the plan, merchant ids and statuses normalised, and any phone number the user never gave is refused.

Then we tested the demo mission itself, in a real browser, many times, and every failure became a rule in code with a test that fails without it:

  • One question per message, many messages. The model asked its opening questions one by one. Now they come together with complete answer sets, and once answered the next call must use a tool (tool_choice: required), so it acts instead of asking again. Questions of taste in the middle of the work are decided, not asked.
  • Another time, accepted alone. A restaurant was full at 8pm and offered 7pm; the agent took it and paid. Now any other time a merchant offers blocks every booking and deposit there until the user taps one.
  • A birthday cake perfume. Asked for a cake, the model searched the product catalogue and found a perfume and a cake tin. Cakes and flowers now always come from a local bakery or florist of the network, delivered to the venue.
  • Long trips. On a five-city rail trip, Nemotron sometimes asked the user to "tap Approve" on a payment card it had never created, and looped. Short missions are reliable; long multi-city trips are what we are fixing next: the payment card must come from code, never only from the model's words.
  • Thinking out loud. "Wait compute: 296.40 + 50…" reached the screen once. Such a message is now rewritten before anyone reads it, as are several answer lines, a title in the wrong language, or wrong arithmetic.

With these rules, in our last six runs of the birthday mission on Nemotron 3 Super, the first PayPal deposit waited for approval 31 to 70 seconds after the request in five runs (99 s in the sixth), and runs that went to the end closed with three bookings, inside the budget, in three to four minutes.

Accomplishments that we're proud of

  • A full money cycle (mandate, hold, capture, void, refund, invoices) driven by an open model on open infrastructure.
  • Guardrails that make any model safer with money, measured on a reproducible bench rather than judged by eye.
  • It works in English and French, by text, live voice or photo, in any city.
  • The demo you can try is the one in the video: every step shown on the right of the video is the agent's real event log on Nemotron.

What we learned

Open models are ready to run agents that touch money, as long as the code keeps the books and the calendar. The fastest wins came from giving the model exact facts (dates, totals) instead of asking it to compute them.

What's next for Mandat

Run the bench on more missions and models, try Nemotron 3 Nano or 3.5 Lightning for the merchant agents (the cheapest calls), and move hosting to Nebius Serverless.

Built With

Share this project:

Updates

Submission history