SycoScan

Inspiration

I'm interested in AI alignment, which is a large branch of AI development which focuses on ensuring AI stays 'aligned' with human values. This hackathon is about building useful applications on Nemotron, and I wanted a project that connected the two.

Sycophancy turned out to be that bridge. It's one of the clearest small-scale examples of misalignment in the systems people use every day. Models are often trained through reinforcement learning from human feedback, and people tend to rate agreement and flattery highly, so a model can learn to optimise for approval instead of truth. The result is an assistant that says "great plan!" when the plan has a flaw. The reply feels helpful but fails the user.

Alignment research on this usually happens in labs and benchmarks. I wanted to bring it to the point where ordinary people meet it: right next to the reply, in a tool anyone can use. If the failure is invisible in the moment, the first step toward dealing with it is making it visible.

What it does

SycoScan tags an AI reply as Independent, Mixed signals, or Likely sycophantic, with a 0-100 score and the evidence behind it.

Instead of guessing from the reply's wording, SycoScan runs a counterfactual test. If 's' is the stance the user expressed, a trustworthy answer should not depend on

$$A(\text{prompt},\, s) \approx A(\text{prompt},\, \varnothing) \approx A(\text{prompt},\, \neg s)$$

SycoScan rewrites the prompt into a neutral version (stance removed) and a flipped version (opposite stance), answers both, and compares them with the original reply. If the conclusion follows the user's opinion rather than the facts, the reply is flagged, and the panel shows all three answers side by side. Agreement with a user who is actually right is not penalised.

You can use it through a Chrome extension that adds a tag under each reply, or through a paste-in web page.

How I built it

The pipeline uses all three Nemotron tiers on Nebius Token Factory, each for the job it suits:

1. Nemotron Nano screens every reply cheaply. If nothing looks suspicious, it stops there, which saves most of the cost.
2. Nemotron Super extracts the user's stance, writes the neutral and flipped prompts, and answers both in parallel.
3. Nemotron Ultra acts as the adjudicator, deciding whether the substance changed because of the stance and writing the plain-language explanation.

The backend is a FastAPI service calling Token Factory through its OpenAI-compatible API. The frontend is a Manifest V3 Chrome extension with site adapters for ChatGPT and Claude.ai, plus a paste-in page as a fallback. The extension makes its network calls from a background worker, and everything shown on a page is inserted as plain text.

Challenges I ran into

• The judge can be biased too. A model that grades sycophancy can be sycophantic itself. Using the counterfactual method means the verdict rests on how answers change, not on one model's taste, though it doesn't remove the problem completely.
• Keeping cost and latency down. Three model calls per check is slow and expensive if done for every reply. The Nano fast path and sending only suspicious replies to Ultra address this.
• Chat pages change constantly. Reading replies from another site's page is fragile, so I added the paste-in page and kept the site adapters small and easy to update.
• Privacy. The tool handles people's conversations, so the extension only sends a reply when the user clicks, and the backend stores nothing.

Accomplishments that I am proud of

• Differential testing is a better way to detect bias than reading one reply in isolation.
• Giving each model tier a distinct job made the system faster and cheaper than sending everything to the largest model.

What I learned

• Sending every reply through the largest model is wasteful. A Nano screen with a fixed threshold lets clearly fine replies exit after one cheap call, and the full Super-then-Ultra path only runs on suspicious ones.
• The answers to the neutral and flipped prompts come from a model that can itself be sycophantic. The neutral prompt reduces that risk because there is no stance to cater to.

What's next for SycoScan

• Verify factual claims in the prompt with web search, so verdicts have outside evidence.
• Measure precision and recall on public sycophancy benchmarks.

Built With

• asyncio
• css3
• fastapi
• html5
• javascript
• nebius-token-factory
• pydantic
• python
• uvicorn