Inspiration
Open-source maintainers can spend hours deciding whether a vulnerability report is real. A negative result is especially hard to trust when it does not say what was tested or how much of the claim was covered. I built Mimi to make that investigation easier to review and to keep the final decision with the maintainer.
What I built
Mimi turns an npm vulnerability report into an evidence dossier. It structures the claim, checks the package version and cited symbols, searches public advisories using identifiers, and proposes bounded test hypotheses with predeclared deterministic oracles. The dossier shows commands, outputs, coverage, and a draft maintainer response. Mimi never auto-closes a report.
There are two ways to explore it. The public replay opens a fixed sample dossier without an account or provider calls. The live application runs the frontend and API and requires a judge access key for live reports. Its bundled sample is still fixture data. For live analysis, report text goes to Nebius Token Factory; Tavily receives only public identifiers, and generated PoC code goes to Nebius Sandboxes when execution is enabled.
How I built it
The backend uses Python, FastAPI, and SQLite; the frontend uses React, TypeScript, Vite, and Tailwind CSS. Nebius Token Factory runs NVIDIA Nemotron 3 Nano, Super, and Ultra. Nano structures claims, Super synthesizes advisory research and drafts replies, and Ultra proposes bounded test hypotheses. Tavily searches public advisory sources. Deterministic backend logic evaluates declared oracles and coverage; incomplete runs and missing evidence stay inconclusive, and grounding alone cannot reject a report.
The combined app is hosted on Render Free, with runtime inference through Token Factory and execution through Token Factory Sandboxes. The public Apache-2.0 repository includes setup instructions and published evaluation results.
What works today
On October 19, a hosted public [email protected] report completed inference, five Tavily queries returning 17 advisory records, eight Sandbox runs, and a saved Needs human dossier and reply without stage errors. It triggered no hypotheses in one configuration. This verifies the integration; it does not establish vulnerability-reproduction accuracy.
Separate E1–E4 pilot runs are published in the evaluation results. E1 had zero fixed-version-confirmed reproductions in 11 cases. E2 did not distinguish real from fabricated reports: none of five fabricated reports was contradicted, and one incorrectly reached Likely. Tavily hit its usage limit during that historical run; funding was restored and later runtime searches succeeded. These small results do not establish reliable triage.
Challenges and lessons
The main design challenge was making a negative result useful without treating missing evidence as proof. Mimi requires declared oracles and records coverage; failed builds and incomplete runs cannot become clean negative results. Language models help structure reports and suggest tests, while deterministic evidence and human review govern the outcome.
The Sandboxes integration initially used an outdated endpoint that returned HTTP 403. After correcting the endpoint and SDK integration, a harmless no-op build/run succeeded, and Mimi's child network namespace blocked an outbound TCP attempt with ENETUNREACH. Live pipeline runs now complete, but provider-level containment of hostile code remains unverified. I keep that limit visible and label the public replay as fixture data.
A five-case routing comparison matched expected tiers for routed, Ultra, and Super settings, but is too small to establish a general quality or cost advantage. Next I want a larger labelled evaluation, better fixed-version reproduction, and a deeper review of the execution boundary.
Log in or sign up for Devpost to join the conversation.