VibeCheck

Main Dashboard
Identified Vulnerabilities
AI analysis and Graphical Representation
API Documentation
Methods to implement API
API Documentation Intro

Inspiration

In the age of vibe coding, a developer can prompt together a working app in minutes, and ship it. But AI-generated code is infamous for it's lack of security: Hardcoded secrets, SQL injection, missing auth, exposed debug endpoints. The AI can't take security into account, and most vibe coders aren't double checking the code.

We wanted to build a tool that fits directly into the vibe coding workflow. Not a separate security platform that adds time and complexity, but an accessible API that agentic IDEs like Cursor can call autonomously while you build.

What it does

VibeCheck is a public REST API with two scanning modes:

Lightweight mode takes a GitHub repo URL or uploaded source files and runs five static analysis modules: dependency scanning against a Gemini-powered contextual code review that catches business logic flaws that regex can't, a curated CVE database, regex-based pattern detection for SQL injection/XSS/command injection, secret detection using entropy analysis, config auditing for Dockerfiles and environment files.

Robust mode utilizes real penetration testing to ensure your product works in the real world. Four AI agents powered by Gemini (recon, auth, injection, config) actually attack your running application with real HTTP requests. They plan their own attack strategies using function calling, send payloads, analyze responses, and report confirmed vulnerabilities with proof.

Any running app is a valid target. Already deployed? Just pass the URL. Running locally? Use Cloudflare Tunnel, ngrok, VS Code port forwarding, or VibeCheck's own built-in WebSocket reverse tunnel.

MCP Integration makes VibeCheck a native tool inside your IDE. We host an MCP server that any compatible editor (Cursor, Claude Code, Antigravity, etc.) can connect to by simply adding our server URL to their mcp.json config file. No setup, no wrapper scripts, no custom HTTP logic. Once connected, the IDE agent gets full access to create scans, read findings, analyze vulnerabilities, check agent logs, and search memory, all through natural language. Ask the IDE to "scan this repo for security issues" and it handles the rest autonomously.

Every finding can be further analyzed with a dedicated AI analysis endpoint that provides root cause, impact assessment, and remediation steps. A semantic memory layer powered by Supermemory lets you search across all historical findings to spot recurring patterns.

How we built it

The API is built with FastAPI and async SQLAlchemy on top of Postgresql. All scanning runs asynchronously in background tasks. The robust agents use Gemini's function-calling API with three tools (http_request, check_headers, report_finding) in a loop, where the model decides what to probe next based on previous responses.

For API access to local builds, devs open a persistent WebSocket connection to the API, and when agents need to hit the target app, requests get serialized as JSON, sent through the WebSocket, executed locally by the client, and the responses come back the same way.

The frontend dashboard is vanilla HTML/CSS/JS served directly by FastAPI. It includes real-time WebSocket status streaming, severity distribution charts, paginated and filterable findings tables, CSV/JSON export, and interactive "Try It" documentation for all 18 endpoints.

We also host an MCP server so IDE agents can call VibeCheck through the Model Context Protocol, giving them native access to create scans, read findings, and trigger analysis without any custom HTTP setup.

Challenges we ran into

The biggest challenge was architecting how robust mode should work. We had to balance developer convenience with practicality. We considered building a custom SDK that devs would need to install, cloning and hosting target apps on the API's end, and several other approaches. Each had tradeoffs: an SDK adds friction, hosting apps server-side is computationally intensive and introduces sandboxing complexity.

We landed on the current approach where robust agents target an open port, and the developer chooses how to expose it. The developer has complete freedom for the method to expose the port. This preserves computing power on the API side while giving developers flexibility to use whatever tool they're already comfortable with.

Our foremost priority was ease of use for prospective developers. Every architectural decision was made with simplicity and modularity in mind. We didn't want VibeCheck to feel like a security platform you have to learn. It should feel like a single API call.

Accomplishments that we're proud of

The robust mode agents actually find real vulnerabilities with proof. During testing against our intentionally vulnerable demo app, the injection agent independently discovered and confirmed SQL injection with a working payload.

The API has 18 endpoints, consistent error handling with typed error codes, pagination/filtering/sorting/full-text search on every list endpoint, idempotency key support, WebSocket streaming, and X-Request-ID tracing on every response.

What we learned

Designing AI agents that interact with real systems is very different from building chatbots. The agents need to be goal-directed, more adaptive than chatbots, and know when to stop and report back. Function calling with Gemini made it possible for agents to behave like actual pentesters.

We also learned how much developer experience matters for APIs. Small things like consistent error envelopes, meaningful error codes, and comprehensive documentation make the difference between a usable and exceptional API.

What's next for VibeCheck

Adding authentication and rate limiting for production use. Building a CI/CD integration so VibeCheck can run automatically on every push. And expanding the agent toolkit so robust mode can test for more vulnerability classes like SSRF, path traversal, and business logic flaws.