BeforeShip

Open-source CLI and MCP launch gate that checks AI-built apps for production blockers before deploy.

Comment

Inspiration

Vibe coding with Cursor, Claude Code, and Codex has unlocked a new era of solo founders shipping real products. But there's a dangerous gap: AI agents are great at making apps that work, and terrible at making apps that are safe to deploy.

We kept seeing the same pattern in indie hacker communities — someone ships a beautiful app built in a weekend with Claude Code, and then discovers their Supabase RLS was never enabled, their Stripe webhook doesn't verify signatures, or their .env file was committed to the repo. Around 45% of AI-generated code contains security vulnerabilities. These aren't bugs — they're invisible production landmines that AI agents consistently skip.

BeforeShip was built to be the last checkpoint before you go live.

What it does

BeforeShip is an open-source launch gate for vibe-coded apps — a CLI tool and MCP server that runs automated pre-deployment checks across your entire codebase before you push to production.

Run npx beforeship scan and it checks:

  • Secrets & Config — Detects committed .env files, exposed API keys, and insecure public key usage
  • Auth — Flags missing API route guards, IDOR patterns, and client-side-only role checks
  • Database — Validates Supabase Row Level Security (RLS) and identifies unsafe storage or migration practices
  • Payments — Ensures Stripe/Razorpay webhooks verify signatures; catches client-side trust in subscription status
  • API Safety — Flags wildcard CORS, missing rate limiting, and absent input validation
  • AI-Agent Specifics — Detects risky agent behaviors: huge rewrites, deleted tests, changes to sensitive files without verification
  • Deployment Readiness — Verifies smoke tests, health endpoints, and environment separation

The MCP server mode plugs directly into Claude Code or Cursor, giving your AI agent real-time access to BeforeShip checks mid-session — so it can self-correct before it ships broken code.

How we built it

BeforeShip was built using OpenAI Codex for code generation throughout the entire build process.

  • CLI: Node.js + TypeScript with Commander.js — Codex scaffolded the entire check runner architecture and AST-based file scanning logic
  • MCP Server: Built on @modelcontextprotocol/sdk — Codex wrote the tool definitions so AI agents can invoke BeforeShip checks directly mid-session
  • Check Engine: Deterministic rule-based scanners for security checks (secrets, auth, DB, payments), with optional OpenAI API integration for stack-aware risk triage and natural language fix suggestions
  • Target Stack: Next.js + Supabase + Stripe/Razorpay + Vercel (the exact stack most vibe-coded apps use)
  • GitHub Action: Optional CI/CD integration to gate deploys on a passing BeforeShip scan

Codex wrote the majority of the scanner logic — the regex patterns, AST traversal for route detection, and the MCP tool schema. We used GPT-4.1 to generate human-readable explanations and fix prompts for each failed check.

Challenges we ran into

  • AST vs. regex tradeoffs: Full AST parsing is accurate but slow; pure regex is fast but misses context. We built a hybrid approach — regex for quick pattern detection, AST confirmation for high-severity findings like auth and payment checks.
  • False positive rate: Early versions flagged too aggressively. We added severity levels (critical / warning / info) and confidence scoring so developers aren't overwhelmed.
  • MCP integration: Making BeforeShip's checks available as MCP tools required careful schema design so Claude Code could invoke them naturally mid-session without breaking flow.
  • Vibe-coded app diversity: Apps built with different AI tools have wildly different structures. We built stack detection to automatically adjust check sensitivity based on the detected framework.

Accomplishments that we're proud of

  • npx beforeship scan works zero-config on any Next.js + Supabase project in under 30 seconds
  • The MCP server is verified compatible with Claude Code, Cursor, and VS Code Copilot
  • Covers the 7 most common production failure patterns in AI-built apps based on real indie hacker post-mortems
  • Built entirely using Codex — a security scanner for AI-built apps, itself built by AI
  • MIT licensed and fully open-source from day one

What we learned

The biggest insight: the production gap isn't a skill gap — it's a tooling gap. Solo founders using AI coding tools have no equivalent of a senior engineer saying "wait, did you check X before shipping?" BeforeShip is that voice, automated.

We also learned that MCP is the right protocol for this — having BeforeShip available as a tool inside Claude Code means the AI agent can proactively run checks and self-correct, not just flag issues after the fact.

What's next for BeforeShip

  • More stacks: Extend beyond Next.js + Supabase to support Django, Rails, and FastAPI backends
  • BeforeShip Cloud: A hosted dashboard where teams can track scan history, compliance trends, and get PR-level diff scanning
  • AI fix mode: beforeship fix --auto — use Codex to automatically patch the most common findings
  • Community check library: Let indie hackers contribute checks for tools they use (PlanetScale, Railway, Clerk, Resend, etc.)

Built With

Share this project:

Updates