ABC Vision Nemotron Coach Agent

A Nemotron 3 Super agent on Nebius that calls real MCP tools and may only state numbers those tools returned. Ungrounded answers are replaced; about a tenth of a cent per question.

Comment

Inspiration

ABC (AI Badminton Coach) is a shipping coaching product whose defining behaviour is refusing to answer when it is not sure: of 705 production jobs that produced a report, it declined to assign a skill level in 42.7%, with a recorded reason each time. The question for this hackathon: can a language model talk about that evidence in plain language without inventing any of it?

What it does

Ask the coach, in plain language, things like "why was no level given?" or "how often does the system decline to publish?". A Nemotron model on Nebius Token Factory decides which read-only tools to call, the tools run on bundled, de-identified data, and the model answers from their results. Each answer is labelled source: "nemotron" (passed the guard) or source: "rules" (fixed fallback text, with a reason). It cannot analyse video or produce a skill level. It is live at https://d2xks2we90iwvv.cloudfront.net/ (MCP endpoint at /mcp). Code (MIT): https://github.com/ekyo0224/abc-vision-mcp

How we built it

  • Model: Nemotron 3 Super (nvidia/nemotron-3-super-120b-a12b) via Nebius Token Factory's OpenAI-compatible API, with OpenAI-style tool calling. Thinking is switched off.
  • Tools: an allowlist of four read-only, in-process tools. Other tool names are refused.
  • Provider-agnostic core: agent_core.py imports nothing from the server; Nebius is one adapter, another OpenAI-compatible host is a configuration change.
  • Facts first: tool results start with plain-English facts, so the model quotes figures rather than computing them.
  • Grounding guard: every number in the final answer must appear in the tool results or the question, and a skill grade is allowed only if a tool returned one. One correction is allowed; then the rules text is returned.
  • Caps: at most 4 tool rounds, 600 output tokens and 20 s per model call, 75 s per question; no retries on errors. Daily cap of 200 model calls; web visitors get 5 a minute and 30 a day; MCP callers share 100 a day; identical questions cached 10 minutes.
  • Click-only: the page loads fixed text; the model is called only from a click or submit.

Challenges we ran into

  • Thinking mode returned empty content. With Nemotron 3's default reasoning on, Nano returned empty content (the token budget was consumed) and fell back to rules text. Turning it off (chat_template_kwargs.enable_thinking=false) made tool calling reliable.
  • Models inventing numbers. In live runs the model computed figures not in the tool output (a new percentage, a minutes conversion); the guard rejected them. After we moved to facts-first tool results, 9 of 9 preset runs passed with no retry.
  • Stray reasoning tags. With Super and thinking off, a reply occasionally still contained a </think> tag; the guard now rejects it.
  • A leaked internal label. An outcome code carried an internal engine value. Review caught it; it was renamed everywhere with a regression test (docs/CORRECTIONS.md entry 5).
  • Model IDs. Casing differs between models; we fixed the default to the real IDs (see feedback).

Accomplishments that we're proud of

A live agent whose output is labelled by origin and checked in code, at about US$0.001 per question.

What we learned

Put the numbers in the tool result and let the model phrase them. A number guard is cheap but does not check meaning.

Honest limits. The guard checks numbers, not wording: an answer can use only returned numbers and still imply a cause or comparison the tools did not give, and the guard will pass it. The prompt forbids this, but that is an instruction, not a check. Rate-limit and cap counters are per worker process, not shared. Tests mock the network; live runs were small manual checks, not a benchmark.

What was significantly updated during the Submission Period

The OpenCV vision pipeline, MCP server and production statistics predate 2026-10-08. All Nemotron and agent work was built 2026-10-08 to 2026-10-12: coach notes (explain_report); the tool-calling agent (ask_coach, POST /demo/ask, the "Ask the coach" box); the provider-agnostic core and Nebius adapter; guard, caps and rate limits; facts-first results; and the model-ID, thinking-off and outcome-code fixes.

How we used NVIDIA Nemotron / Nebius Token Factory

  • Default agent model: nvidia/nemotron-3-super-120b-a12b. Nano (nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B) also calls tools correctly, is cheaper, and is selectable via NEBIUS_AGENT_MODEL.
  • Prices read from /models?verbose=true on 2026-10-12 (per 1M prompt / completion tokens): Super US$0.30 / 0.90; Nano US$0.06 / 0.24.
  • Measured live, 6 questions on Super: about 3,100 prompt and 220 completion tokens, roughly US$0.0011 per question. Separately, 6 of 6 live coach-notes calls (Nano path) on 2026-10-12 validated as nemotron.

Feedback on Nebius Token Factory and NVIDIA models

  • Model ID casing is inconsistent. The Nebius cookbook page for Nano uses nvidia/nvidia-nemotron-3-nano-30b-a3b, but GET /models on our account lists nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B, and IDs are case-sensitive: the lowercase one failed and fell back. The adapter now checks the chosen model against /models and falls back to a listed one.
  • Thinking-on default hurts tool calling. For Nemotron 3, reasoning is on by default and Nano returned empty content when max_tokens was consumed. A note beside the tool-calling examples would save time.
  • /models?verbose=true pricing was genuinely useful. It let us compute per-question cost from the account's own listing.

Demo

Built With

Share this project:

Updates

Submission history