Scavibe

Inspiration

Vibe coded apps always have the same dangerous patterns that non tech developers always missed: getting an app working is easier than ever, but checking whether it is safe, scalable, and ready for real users is still hard.

A solo founder can ship in a weekend, then discover too late that an API route has no authorization check, a secret was committed to the repository, or the app slows down as soon as traffic arrives. A proper review can be expensive and slow, so many early products launch without one.

I built Scavibe to give founders a practical pre-launch check. Instead of vague AI advice, it shows what was found, where it was found, why it matters, and what to fix next.

What it does

Scavibe takes a GitHub repository and guides it through three audit stages.

1. Performance

Scavibe load-tests a disposable sandbox deployment rather than sending traffic to a founder’s production site. It ramps through fixed concurrency levels, records response times, error rates, completed requests, and identifies the first confirmed breaking point.

The report explains the tested range, the measured result, and the changes that would improve reliability. Performance findings only appear after a qualifying confirmation test, not from a short exploratory spike.

2. Security

The Security stage reviews the supplied repository snapshot against an OWASP-focused checklist. Every accepted finding must include an exact file path, line range, and source quote from the pinned commit.

Scavibe calculates severity and confidence in code, rather than letting the model assign them. Where a safe, read-only sandbox proof of concept is possible, the report records the proposed check, execution result, HTTP status, response evidence, and final status:

• Confirmed exploitable
• Candidate — unconfirmed

This makes it clear which issues were demonstrated and which still need developer review.

3. Legal and data handling

The Legal stage traces evidence of data collection, consent flows, privacy links, age-related safeguards, analytics, and related product behavior. It highlights gaps for the jurisdictions selected by the user and produces a structured data-handling report.

Scavibe does not present legal output as legal advice. It labels the results as a review aid and recommends attorney review for final policies or legal decisions.

All three stages export PDF reports. After the audit, founders can download the complete set of Performance, Security, and Data-handling reports together.

Scavibe never pushes directly to a user’s repository. Any approved output is prepared for review through a pull request workflow.

How we built it

Scavibe is built with a Next.js frontend and a FastAPI backend.

The frontend provides a guided audit flow with animated progress states, stage timelines, Server-Sent Events, and downloadable reports. The backend pins an audit to a specific GitHub commit so the evidence always points to the exact code version that was reviewed.

For performance, we built a bounded sandbox-only HTTP load-testing workflow. It runs exploratory steps across nine fixed concurrency levels, detects a possible breach from measured latency or error rate, and then performs a longer confirmation run before reporting a performance finding.

For security and legal analysis, we built separate specialist agents with their own prompts, validators, and evidence rules. The model can suggest candidate findings, but Scavibe only accepts them after the backend confirms that the cited file exists, the line range is valid, and the quoted code matches the pinned source.

Where Codex and GPT-5.6 accelerated me

• Codex helped me build the full-stack application: the Next.js interface, FastAPI audit pipeline, typed data contracts, Server-Sent Events, PDF exports, sandbox safety controls, and unit tests.
• Codex was especially useful while handling edge cases that matter in a real audit product: malformed model JSON, incorrect line citations, failed requests during load testing, and safe sandbox boundaries.
• GPT-5.6 Terra is Scavibe’s primary specialist-analysis model. It converts the pinned repository context into structured candidate findings for Performance, Security, and Data-handling review.
• The model does not get the final word. Scavibe verifies source evidence, calculates scoring deterministically, enforces performance thresholds, and blocks unsafe proof-of-concept requests in backend code.
• When the OpenAI API is unavailable or rate-limited, Scavibe can use a clearly labeled NVIDIA fallback. Reports identify the engine that actually produced the analysis, so fallback output is never presented as GPT-5.6 output.

Key decisions

• I chose sandbox deployment for load testing because sending traffic to a founder’s live site can create surprise costs or look like a denial-of-service attack.
• I chose exact source evidence over broad claims. A finding without a valid file, line range, and quote is rejected.
• I chose confirmation testing for performance results. A brief spike is not enough to claim an application will fail under load.
• I chose pull requests over direct pushes so the founder remains in control of every change.
• I chose to label legal output as a review aid, not legal advice.

Challenges I ran into

The hardest part was making the agents trustworthy instead of simply making them sound confident.

Models sometimes return malformed JSON, use the wrong field names, cite a nearby line instead of the exact source range, or describe a risk without enough evidence. We built a strict validation layer around the model output. If a citation does not match the repository snapshot, Scavibe rejects it and gives the agent one chance to repair the response. It never silently changes or invents evidence.

Sandboxed performance testing also required careful handling. We needed to separate short exploratory measurements from qualifying evidence, avoid traffic to live production systems, handle zero-success responses honestly, and confirm a suspected breaking point before reporting it.

Finally, I worked on making the product understandable for non-technical founders without hiding the technical detail developers need. The UI gives a clear progress story, while the reports retain evidence, measurements, remediation steps, and limitations.

Accomplishments that I'm proud of

• Built a complete three-stage pre-launch audit flow for performance, security, and data handling.
• Created sandbox-only load testing with measured ramp results and confirmation runs.
• Made source findings verifiable through exact file, line, and quote validation.
• Added safe proof-of-concept tracking with clear confirmed versus unconfirmed statuses.
• Built downloadable PDF reports for every completed stage.
• Kept automated changes reviewable through pull requests instead of direct repository pushes.

What I learned

I learned that the useful part of AI in developer tooling is not just finding issues. It is making each result concrete enough for a founder or developer to verify and act on.

I also learned that safety boundaries need to be part of the product itself. Testing sandbox deployments, validating every citation, separating confirmed findings from candidates and keeping people in the pull-request loop made Scavibe more credible and more useful.

What's next for Scavibe

Next, I want to support private repositories, expand data-handling coverage across more jurisdictions, and add continuous monitoring so projects can be re-audited after each commit instead of only before launch.

I also want to expand the reviewed remediation workflow so founders can move from evidence-backed finding to a safe, reviewable pull request with less friction.

Built With

• css
• fastapi
• nextjs
• python
• typescript