Sherpa

Inspiration

AI coding assistants hand learners the answer the moment they get stuck. The productive struggle which drives learning becomes destroyed by this approach. A skilled human tutor performs an initial assessment to determine if the student continues to learn or if they have reached their breaking point and if providing a hint would lead to the correct answer. We wanted a tutor that does that weighing out loud.

What it does

Students work on Python problem solving by solving 31 LeetCode-style Easy and Medium problems which include Two Sum and Valid Parentheses and Climbing Stairs. Users can operate code directly through their browser interface. Sherpa monitors all work activities by tracking both code changes and testing activities and breaks and all communication in the chat.

The system activates three peer agents to determine the correct assistance level following any event which includes failed runs and extended idle periods and incoming chat messages and users clicking "Want a nudge?" and scheduled background check-ins. The learner receives live access to all round streams while they watch every agent's argument presentation and their defeated arguments in the rebuttal process. The system provides users with their exact agreed-upon level along with an easy-to-understand explanation about Sherpa's selection process.

Sherpa provides higher-level hints by showing learners the complete execution path of their own code. The system creates call trees for recursive problems through single-call drawings which identify all repeated subproblems to demonstrate how memoization eliminates these duplicates.

How we built it

Deliberation protocol. In round 0 each agent proposes a level. In up to 3 more rounds, agents revise against their peers, moving at most one level per round. Stop rules: consensus acts at that level. A spread of 1 takes the lower, less intrusive level. A spread of 2 or more is a deadlock, so Sherpa withholds the hint and asks "Want a nudge?". Integrity's cap clamps everything. If nobody moves in a round, it stops early. Jev from TypeSafe makes the judgment calls: it scores each agent's position against described levels, and it decides whether a peer's argument actually holds against what this agent observes. Arguments that hold move the agent one level. Arguments that fail become rebuttals shown in the UI. Code owns the hard facts (counters, thresholds, hint timing), and Jev owns the semantic judgments. LLMs only write, never decide. NVIDIA Nemotron (through any OpenAI-compatible endpoint), with Google Gemini as fallback and offline templates as a last resort, words the agents' arguments and writes the final hint. Jev checks every hint twice: it must claim nothing the code and test results don't show, and it must not leak the solution. A rejected hint is rewritten at the same level, then one level lower, then replaced by a template. Explanations. Code writes the first sentence (the test result) and the last (what Sherpa does). The LLM writes only the "why" in between, from a list of deliberation facts, and Jev checks it against those facts. AGNTCY. Every round and decision is published to an end-to-end encrypted (MLS) SLIM group channel, so an instructor dashboard or audit logger can watch deliberations live. Each agent has an OASF record (domain: education/e_learning; skills: negotiation resolution, multi-agent coordination). Redaction. Reference solutions are scrubbed from everything that leaves the system: logs, the bus and SLIM. Backend. Python and FastAPI over a WebSocket. A stdlib state builder turns raw browser events into features, such as "rewrote the same function 3 times", so agents don't have to infer them from logs. Every deliberation is logged as JSONL for replayable evaluation. 124 pytest tests. Frontend. React 19, TypeScript and Vite, with the Monaco editor, Pyodide running Python in a web worker, a live agent panel, chat, a code tracer and a recursion call-tree viewer.

Challenges we ran into

Keeping the reference solution hidden from each LLM yet allowing an agent to reason regarding it. We resolved this via a text-free Integrity agent, performing Jev leak checks and redaction on all outputs.

Preventing LLMs from silently choosing aid levels. We divide tasks so code and Jev determine outcomes while the LLM merely composes the text.

Making disagreement useful. Lacking halt mechanisms, agents might debate endlessly, thus stalling forces return selection authority to the student.

Streaming multi- live round deliberations go to the browser and also to an encrypted SLIM channel at once.

Accomplishments that we're proud of

Fully transparent tutoring. The learner sees why they got the hint they did, including the arguments that lost.

Hints are double-checked for false claims and solution leaks before anyone sees them.

Works offline with template fallbacks, so the demo never depends on an API being up.

AGNTCY integration: encrypted live observation over SLIM and OASF records for each agent.

What we learned

Leave secrets out of the model as well as out of the prompt: we were able to structurally isolate the reference solution (using a text-free Integrity agent, leak detection and redaction) where the instructions had failed.

Keep the decision-making separate from that of writing: after coding and Jev had taken charge of everything that needed to be decided, the LLM was assigned the job of writing, making Sherpa predictable, easy to test and explain.

Disagreement needs to have an escape route: the stop command turned protracted discussions into a signal and a stalemate usually meant that the learner was to make a choice.

Transparency is a systems-level problem and not just a UI design problem: by treating each decision-making process as an event stream we got a live panel, a SLIM observer stream and traceable logs in one implementation.