Kairos

Kairos understands your data. Jenny, its voice companion, understands your business: she joins your meetings, answers from live data and shows the chart. Runs on NVIDIA Nemotron via Nebius.

Comment

Inspiration

Every business has someone asking "how are we doing today?", and the answer is buried in a warehouse of hundreds of tables and dashboards nobody watches. AI agents can query every one of those tables, but they still guess which one is true. We wanted a colleague who watches the numbers all day, knows what normal looks like, and can join your meeting and answer out loud, with numbers you can trust.

What it does

Kairos is the context layer between your data and every AI agent:

  • It maps every table, pipeline and column in your warehouse (Snowflake, Databricks).
  • It turns them into data products the business understands (Sales, Delivery Performance, Customer Retention…).
  • It checks every number with code: provenance, trust, freshness, lifecycle and attestation.

Agents reach it over MCP or a REST context API.

Jenny is the companion built on Kairos:

  • She watches the business's vitals all day against what's normal for that hour and weekday, and tells you first when something moves.
  • She joins your Google Meet as a participant. You ask out loud, she answers out loud in plain numbers, and she puts the chart on her own video tile as she speaks.

In the demo she runs on a food-delivery dataset: 29 tables in Snowflake, mapped and verified by Kairos.

How we built it, and where Nemotron fits

  • Nemotron does the thinking. Jenny's reasoning and every word she says run on NVIDIA Nemotron 3.5 Super-VL, served by Nebius Token Factory through its OpenAI-compatible API. Nemotron plans the lookups, calls Kairos's tools (compare to normal, query a data product, put a chart on screen) and writes what she says. Thinking is switched off for spoken turns (chat_template_kwargs.enable_thinking=false), so the first token arrives in about a second.
  • One setting. LLM_PROVIDER=nebius moves Kairos's own reasoning (enrichment, data products, root-cause analysis, evals) to Nemotron as well, and COMPANION_LLM=nebius:<model> picks Jenny's model. Structured output goes through function calling. Embeddings stay on text-embedding-3-large, so the stored vectors remain valid.
  • We benchmarked every Nemotron on our key with a real streamed tool turn (python -m kairos_kb.llm.nebius_check):
    • Lightning: about 1.0 s to first token.
    • Super-VL: 1.1 s.
    • Ultra: 1.3 s.
    • Super 120B: asked for clarification instead of calling the tool.
    • Nano: about 40 s.

With thinking on, Super-VL was the only one that planned every lookup a profiling question needed in a single turn.

  • Her meeting body:
    • A Playwright-driven Chromium joins the Meet.
    • Meeting audio streams to realtime speech-to-text, and her words are ready about 0.7 s after you stop.
    • A quick read of each turn decides whether it's for her and whether it needs data. She acknowledges at once ("mm-hm, one sec") while Nemotron works.
    • A claim gate checks every sentence against what her lookups returned before it's spoken.
    • Her charts are drawn on the canvas that is her camera.
  • Stack: Python, FastAPI, SQLAlchemy, Postgres with pgvector, Docker Compose, and a Next.js frontend. The demo film was built in Remotion from the real UI.

Challenges we ran into

  • Tool calls written as text. Nemotron Lightning sometimes wrote <tool_call><function=…> as plain text instead of making the call, and Jenny read the markup aloud. The stream now keeps tool-call and <think> blocks out of her speech and turns text-form calls (XML or JSON style) into real ones.
  • Silence on the last step. After using her lookup budget, Lightning kept writing one more call even with no tools offered. The answer came out empty, and she said "I lost my thread" in 6 of 27 spoken answers during testing. The final step now gets no tools, sees its lookups as plain notes rather than tool messages, and has a last retry under minimal instructions.
  • A number about the wrong thing. Asked about McDonald's outlets, Lightning read out the count for all restaurants. Every lookup result now says what it covers (its filters), and her spoken rules say a number is only about its scope. Super-VL followed this ("I don't have any delivered orders for Pizza Hut in Mumbai today"); Lightning didn't reliably, so Jenny runs on Super-VL.
  • Loosely written arguments. Lists sent as "[cancel_rate]", filters as {dimension: city, …}, sort orders quoted twice. These are now repaired before the lookup runs, so a retry isn't wasted.
  • A cropped camera. Meet fills a wide stage by cropping the top and bottom of a 16:9 video, which cut off chart titles and axes. Her charts now draw inside the band that survives.
  • Web search, switched off. We wired in Tavily for news context, then turned it off by default. Next to a spike in the numbers, a headline invites a cause the data can't back.

What we learned

  • Small, fast models can still lose the thread. For a voice agent, the right model is the one that keeps every number tied to what was actually measured.
  • Open models need a little plumbing around tool calling. Once it's there, changing providers is a setting.
  • In front of a room, "I don't have that" beats a confident wrong number.

What we built during the hackathon

Kairos and the first version of Jenny existed before the submission period. During it, we:

  • Added NVIDIA Nemotron on Nebius Token Factory as an env-based provider for both Kairos and Jenny, plus a benchmark tool for every Nemotron on the key.
  • Made Nemotron's tool calling reliable in a live voice loop: hidden reasoning, text-form calls, the final-answer step, argument repair, and scope on every lookup.
  • Moved meeting hearing from Meet captions to realtime speech-to-text (from 1.2–2.2 s down to about 0.7 s), added natural listener sounds, and fixed how she tells when she's being addressed.
  • Fixed her camera for wide Meet tiles.
  • Recorded the demo with Jenny on Nemotron 3.5 Super-VL.

Built With

Share this project:

Updates

Submission history