ShipProof

• 

  ShipProof: Proof Before Done. Verify software completion claims with test evidence and expose missing proof.

Inspiration

AI coding agents can write code, run tests, and claim a task is complete. But passing tests do not automatically prove that every requirement has been implemented correctly. ShipProof was built around one question: What evidence actually proves that software is ready?

What it does

ShipProof is a local-first tool that audits software completion claims against test evidence. It links explicit requirements to test results and classifies them as VERIFIED, FAILED, or UNVERIFIED. It identifies missing evidence and failed checks instead of treating a passing test suite as proof that everything works. The prototype supports JUnit XML test-result imports, requirement-to-test mapping, a claim audit with blockers, and JSON readiness-report exports.

How we built it

We built ShipProof using TypeScript, Vite, and a browser-based interface, with Vitest for automated testing. The evaluation logic checks whether requirements have relevant linked test evidence and whether those checks pass. The interface displays the audit decision and its blockers, while the JSON export provides a portable record of the same assessment. The application runs locally without requiring a backend service.

Challenges we ran into

The biggest challenge was distinguishing a passing test from relevant evidence. For example, a passing login test does not prove that rate limiting is enforced. A requirement should remain unverified when its supporting evidence is missing, even if other tests pass. We also needed to keep the displayed audit and exported JSON consistent so that the report does not contradict the interface.

Accomplishments that we're proud of

• Built an end-to-end prototype for auditing software completion claims.
• Implemented JUnit XML evidence import and requirement-to-test mapping.
• Made missing evidence visible instead of silently treating it as success.
• Added a claim audit with explicit blockers and a JSON readiness report.
• Designed conservative verification rules that distinguish successful checks from unsupported claims.

What we learned

We learned that execution is not verification. A test passing is useful evidence, but only when it meaningfully supports the requirement being evaluated. We also learned that a trustworthy engineering tool must communicate uncertainty honestly. Marking something unverified is more useful than presenting confidence that the available evidence cannot justify.

What's next for ShipProof

Next, we want to explore integrations with CI pipelines and AI coding agents, additional evidence formats, and automated detection of requirements that lack adequate test coverage. Our goal is to make software completion claims more transparent, traceable, and evidence-based.

Built With

• api
• css
• dom
• github
• html
• javascript
• json
• junit
• npm
• testing
• typescript
• vite
• vitest