Second Thought is an evidence-first control layer for AI agents. It makes visible when an agent should acquire more evidence, increase reasoning effort, preserve uncertainty, or hold a tool request before dispatch.
The application combines an adaptive controller that uses NVIDIA Nemotron models through Nebius Token Factory with an optional Tavily retrieval path. It records policy transitions and correlated run receipts. A separate Python MCP execution adapter binds the arguments that were evaluated to the arguments actually dispatched. It can hold an unsupported write before it reaches a synthetic sandbox backend, while the backend retains its own eligibility checks.
The workspace includes four controlled examples: Berth 7 operational diagnosis, Run 440 experimental-control review, Door 2 access-device diagnosis, and a synthetic refund boundary. The Evaluation page compares direct answering, generic self-critique, and Second Thought on a fixed 12-case pilot with two repeats per case and 72 planned executions. The latest detector revision completed 71 of 72 cells. Final passes were 13/24 for direct answering, 19/24 for generic self-critique, and 18/24 for Second Thought. A narrow difficulty-framing false intervention on PT-19 moved from 2/2 to 0/2 after the detector revision. Overall answer accuracy did not improve, and this pilot does not establish superiority over generic self-critique. Reports retain failed cells and distinguish deterministic scoring from model-judged scoring.
A separate frozen 20-case NMM pressure suite is provided as an evaluation asset, not a fine-tuning dataset. A hash-verified live Berth 7 receipt records three Nemotron 3.5 Lightning calls, one Nemotron 3 Super call, one Nemotron 3 Ultra call, and one Tavily retrieval. The adaptive run acquired evidence, escalated and de-escalated model effort, and committed a targeted examination result.
Second Thought is a bounded research prototype. The judge-facing experience uses preloaded evaluation reports and saved receipts so reviewers can inspect evidence without purchasing API usage. Live provider execution remains owner-controlled and is documented separately. The project does not claim production non-bypassability, broad detector reliability, or universal reasoning superiority.
Built With
- nebius-token-factory
- nvidia-nemotron
- react
- tanstack-start
- typescript
- vercel
- vite