BlackBox — Evidence-Validated Security Assessment for Pull Requests
Reproduce the security failure, validate the prevention, and verify the result before merge.
Inspiration
A pull request can look harmless, pass its normal tests, and still introduce a serious authorization or privacy weakness.
Traditional scanners and AI code reviewers often stop at a warning:
“This code may contain an authorization vulnerability.”
That warning may be useful, but it still leaves the developer with important unanswered questions:
- Can the weakness actually be reproduced?
- What information could be exposed?
- Which authorization boundary is incomplete?
- What code change would prevent the issue?
- Does the proposed fix block the original failure?
- Does the fix continue to work across many related test cases?
- Were existing product features preserved?
BlackBox was built to close that gap.
What it does
BlackBox converts a security-sensitive pull request into an evidence-backed repository assessment.
For the reference scenario, BlackBox evaluates a cross-tenant customer export authorization weakness.
It:
- Reviews the changed code and relevant authorization boundaries.
- Produces a structured security hypothesis.
- Maps the weakness to validated security classifications such as CWE and OWASP API Security.
- Reproduces the unauthorized behaviour using controlled synthetic data.
- Records runtime evidence and source references.
- presents a reviewable prevention change.
- Re-runs the original request after prevention.
- Tests hundreds of related authorization variations.
- Confirms whether existing feature tests still pass.
- Exports a portable Evidence Validation Package and offline HTML report.
The reference assessment demonstrates a measurable result:
Before prevention
- Unauthorized response: HTTP 200
- Synthetic records exposed: 27
After prevention
- Unauthorized response: HTTP 403
- Records exposed: 0
- Generated cross-tenant variations blocked: 500 out of 500
- Existing feature tests: passed
BlackBox does not promote a model-generated statement directly into a confirmed finding. The final status is based on recorded evidence, schema validation, and deterministic verification.
Main capabilities
- Repository security assessment
- Structured authorization analysis
- CWE and OWASP API Security classification
- Attack scenario simulation
- Runtime evidence capture
- Evidence and provenance tracking
- Reviewable prevention workflow
- Before-and-after verification
- 500-case authorization variation testing
- Policy compliance status
- Threat technique profile
- Controlled comparative evaluation
- Evidence Validation Package export
- Standalone offline HTML report
- Resettable and repeatable reference assessment
How we built it
BlackBox was developed as a TypeScript monorepo with a browser application, command-line workflow, shared schemas, test infrastructure, CI configuration, and report-generation components.
The project uses:
- TypeScript
- Node.js
- pnpm
- Zod
- Vitest
- Playwright
- fast-check
- Git
- GitHub Actions
- OpenAI Responses API
- GPT-5.6
- Codex
Strict schema validation is applied at the major data boundaries so malformed or unsupported results cannot enter the user interface.
The guaranteed judge workflow uses a prevalidated reference dataset with synthetic information and does not require an API key.
An optional API-assisted mode is also implemented. It reads OPENAI_API_KEY only from the server-side environment.
How Codex was used
Codex accelerated the implementation and refinement of:
- the monorepo structure,
- browser workflow,
- command-line interface,
- schemas,
- automated tests,
- CI configuration,
- evidence exports,
- documentation,
- terminology audit,
- startup verification,
- clean-install verification,
- release preparation.
Codex also helped identify defects, update tests, validate the final browser workflow, and prepare the project for judge evaluation.
Human decisions remained responsible for:
- product scope,
- security boundaries,
- evidence requirements,
- human-review controls,
- terminology,
- supported claims,
- limitations,
- final release decisions.
How GPT-5.6 was used
GPT-5.6 supports structured repository and security reasoning through the OpenAI Responses API integration.
Its responsibilities include:
- interpreting relevant code changes,
- identifying protected assets,
- identifying authorization boundaries,
- proposing testable security hypotheses,
- referencing supporting code,
- suggesting validation strategies,
- producing structured output for the application.
Model output is validated before it enters the product.
GPT-5.6 does not independently decide that a finding is confirmed or prevented. Recorded runtime evidence and deterministic checks determine the final assessment status.
Challenges we faced
Building a complete verification workflow
The main challenge was moving beyond a warning-only experience. The system needed to connect code analysis, controlled reproduction, evidence capture, prevention review, repeated verification, and final reporting.
Maintaining an honest evidence boundary
We designed the product so that model confidence cannot automatically become a confirmed security result. A finding must move through explicit stages such as hypothesis, code evidence, reproduction, prevention, and verification.
Creating a reliable judge workflow
The project needed to work without credentials or manual configuration. We created a repeatable reference assessment using synthetic data so judges can test the primary product experience immediately.
Keeping the interface technically precise
We completed a repository-wide terminology audit to remove ambiguous, promotional, prototype-oriented, and misleading language from the customer-facing interface.
Accomplishments
We are proud that BlackBox:
- completes a full repository assessment workflow,
- demonstrates a real before-and-after authorization result,
- blocks 500 out of 500 generated cross-tenant variations,
- preserves existing feature tests,
- records evidence and provenance,
- validates exported evidence integrity,
- exports a standalone offline report,
- runs without an API key in reference mode,
- includes a documented optional OpenAI integration,
- passed automated CI and manual browser acceptance testing,
- provides a clear testing path for judges.
What we learned
We learned that AI-assisted security analysis becomes more useful when its output is connected to structured evidence and deterministic validation.
The model is valuable for understanding code, identifying security boundaries, and proposing testable hypotheses. However, the most important product decision was to separate model reasoning from final evidence status.
We also learned that judge usability matters as much as technical depth. A strong developer tool must be easy to install, easy to test, honest about its limitations, and clear about which results are generated, recorded, or independently validated.
Security and privacy
- The public demonstration uses only synthetic data.
- API credentials remain server-side.
- No API key is included in the repository or video.
- Repository content is treated as untrusted input.
- Arbitrary repository command execution is not enabled by default.
- Evidence exports include integrity checks.
- The project does not claim enterprise-grade signing unless real signing is implemented.
- The project does not claim full kernel or container isolation where it has not been established.
- Human review remains part of the prevention workflow.
Known limitations
- The guaranteed judge path uses a prevalidated reference dataset.
- API-assisted assessment requires the tester to provide their own server-side
OPENAI_API_KEY. - The current release focuses on a controlled TypeScript authorization scenario.
- Full multi-language support is not included.
- Enterprise-grade workload isolation and managed cryptographic signing remain future work.
- The project does not automatically merge or deploy generated changes.
What is next
Future development will focus on:
- broader TypeScript framework support,
- Python and Go repository analysis,
- stronger container-based execution isolation,
- live GitHub pull-request integration,
- organization policy configuration,
- additional authorization and privacy scenarios,
- managed signing for evidence packages,
- improved support for private local-only analysis,
- expanded continuous integration integrations.
Final result
BlackBox gives developers more than a warning.
It provides a reproducible repository assessment, supporting evidence, a reviewable prevention workflow, and measurable verification before a sensitive change reaches production.
Log in or sign up for Devpost to join the conversation.