Blindfold

Inspiration

Benchmark contamination is one of those problems everyone in AI acknowledges and nobody has fixed. The moment you publish a test set, it gets scraped and lands in the next model's training data. MMLU, GSM8K, HumanEval have all been shown to leak. So a model scoring 95% might be genuinely capable, or might have seen the answers — and there is no way to tell from the outside.

The usual response is to keep the answer key private and run the evaluation yourself. But then the leaderboard is only as trustworthy as whoever operates it, and the people running evaluations often have a stake in the results.

That felt like a false choice, and exactly the kind of false choice zero-knowledge proofs exist to remove.

What it does

Blindfold is a benchmark whose answer key is never published, and a leaderboard anyone can still verify.

An author commits to a 16-question answer key. What lands on the ledger is a single opaque 32-byte commitment, the answers themselves stay on the author's machine. Models then submit answers and are graded by a zero-knowledge circuit that computes the score against the committed key without ever revealing it.

The result is a leaderboard where the scores are provably correct and the test set never leaks. The benchmark stays uncontaminated because there is nothing to contaminate it with.

How we built it

Two circuits in Compact, about 110 lines total:

publishBenchmark commits to the answer key using persistentCommit with a salt, so the commitment hides the answers even though there are only 4^16 possible keys.

gradeSubmission proves a score is correct. It refuses to accept any answer key except the one that opens the commitment already on the ledger, counts the matches itself, and writes out only the score.

Both hashes are domain-separated (blindfold:answer-key:v1 and blindfold:graded:v1) so no value can be replayed into the other position, and the key commitment binds the benchmark ID so one key cannot be quietly reused across two benchmarks.

The web app is React and Vite. It runs the compiled circuits directly in the browser through @midnight-ntwrk/compact-runtime and WebAssembly — the buttons are not calling a mock, they are executing the same compiled contract the tests run.

Challenges we ran into

The design problem was harder than the code. The obvious version of this project — commit to the key, prove the score — is broken, and it took a while to see why. If a submitter can resubmit freely, they flip one answer, resubmit, and watch whether the score moves. Repeat sixteen times and the leaderboard has handed them the entire answer key, one question at a time. The whole scheme leaks through its own output.

The fix is a nullifier capping each model at one graded submission per benchmark. It is two lines, and without them nothing else in the project matters.

On the implementation side, Compact has no mutable local variables, so the score cannot be accumulated in a loop — the tally had to become a single expression across all sixteen questions. The compiler was also strict about disclosure in a way that turned out to be teaching something real: persistentCommit takes a salt and hides its preimage, so inserting the result needs no disclose(), while persistentHash is unsalted and the compiler requires you to declare the disclosure explicitly.

Accomplishments that we're proud of

The attack analysis. Most privacy projects stop at "the secret is hidden", the interesting question is whether the system leaks the secret through its own outputs, and here it very nearly did.

Beyond that: 14 tests against the real compiled circuits covering both attacks, a browser app that executes those same circuits rather than a simulation of them, and a ledger inspector that does not merely claim the answer key is absent but concatenates all public state and checks.

What we learned

That a privacy design is only as good as the attacks it survives, and that the dangerous leak is usually not the obvious one. Hiding the answer key was easy. Noticing that a public leaderboard is itself an oracle that gives up the key took longer.

Also, concretely, how Compact's disclosure analysis works, and why it forces you to be explicit about the difference between a salted commitment and a bare hash.

What's next for Blindfold

Deploy to Midnight preprod with wallet-signed submissions Signed model identities, so a submission cannot be falsely attributed Larger question sets and partial-credit scoring Let an author prove key freshness, so a benchmark can be shown to predate a model's training cutoff Honest note on the current state: the circuits are real and compiled, proving and verifying keys are in the repo, and the app, demo, and tests all execute them. The ledger is in-memory rather than the Midnight testnet: every contract rule is enforced exactly as written, but wallet integration, proof-server proving, and transaction submission are not wired up. Preprod deployment is the next step, not a claim being made here.

Built With

• compact
• midnight
• react
• typescript
• vite
• vitest
• zero-knowledge-proofs