vibecheck

Your candidates will use Cursor and Claude on day one. Why test them without it?

Comment

Inspiration

Traditional programming assessments test memorizing algorithms and solving puzzles in isolation. This does not reflect how engineers write code anymore. Instead, developers use ai tools like Cursor and Claude code. It's time to assess what actually matters which is how candidates think, prompt, debug, and build with AI. Because that's the future of software engineering.

What it does

VibeCheck is an AI-powered coding assessment platform that tests candidates on real-world software engineering skills rather than algorithmic puzzles.

Key Features:

  • Full IDE Experience - Candidates code in a WebContainer-powered browser IDE with file management, terminal access, and live preview

  • AI-Assisted Coding - Tests how well candidates leverage AI tools (like Cursor/Copilot), mirroring modern development workflows

  • AI-Generated Realistic Challenges - Claude creates browser-based coding problems that simulate actual product features (e.g., build a recipe manager, create an e-commerce cart)

  • Holistic AI Grading - Claude Sonnet 4.5 evaluates submissions based on framework-specific rubrics, code quality, UI/UX, and effective AI utilization

  • Time-Travel Debugging - Records every keystroke, AI interaction, and file change for session replay—see HOW candidates think and problem-solve

  • Detailed Feedback - Provides category-by-category scoring, strengths, areas for improvement, and AI effectiveness analysis

The Flow: 1) Candidate selects their preferred framework (React Router v7 or Next.js) 2) AI generates a realistic, practical coding challenge 3) Candidate builds the solution in the browser IDE with AI assistance 4) AI grades their ability to code with AI 5) Employer reviews detailed results and can replay the entire session

How we built it

frontend Stack: Next.js 16 with App Router for the application framework WebContainer API for running a full development environment in the browser Monaco Editor for code editing with syntax highlighting XTerm.js for terminal emulation TailwindCSS + shadcn/ui for beautiful, responsive UI

Backend Stack: Next.js API Routes for serverless endpoints Drizzle ORM with PostgreSQL (Neon) Better Auth for authentication

AI Integration: OpenRouter with Claude Sonnet 4.5 for challenge generation and grading AI SDK from Vercel for streaming AI responses

Challenges we ran into

  • WebContainer Limitations - Running a full Node.js environment in the browser had constraints. We had to filter out node_modules, .next, and build artifacts when collecting code for grading.

  • Database Relations - Drizzle ORM required explicit relation definitions for nested queries. We had to properly define assessmentSubmissionRelations and assessmentSessionRelations.

  • Filesystem Implementation - The filesystem is implemented with the zen-fs library. However, the webcontainer also has it's own filesystem, and the editor ui needs to maintain a copy of the file tree. Keeping all three in sync was a challenge. Especially when it came to editing the files.

  • Race Conditions in Rubric Generation - The rubric took 40+ seconds to generate, causing grading to fail if users submitted too quickly. We implemented a retry mechanism with 50-second polling intervals.

  • Markdown Rendering - Initially used Streamdown but it failed silently. Switched to react-markdown for reliable rendering of coding challenges.

  • AI Response Parsing - AI sometimes returned malformed JSON. We implemented robust regex extraction and error handling to parse grading results.

Accomplishments that we're proud of

  • Built a full WebContainer IDE that runs entirely in the browser—no backend servers for code execution!
  • Created an AI-powered assessment pipeline from challenge generation → IDE coding → automated grading
  • Framework-specific evaluation that adapts grading rubrics based on whether candidates use React Router v7 or Next.js

What we learned

AI is the future of technical evaluation - Traditional tests are outdated; AI can assess code quality, architecture, and problem-solving holistically Browser capabilities are incredible - WebContainers let us run Node.js, package managers, and dev servers entirely in the browser Real-world tests are more valuable - Building a recipe manager reveals more about a candidate than solving "Two Sum" AI-assisted coding is a skill - How well someone prompts and leverages AI is as important as raw coding ability Event sourcing enables powerful features - Recording state changes as events unlocked session replay without complex infrastructure

Built With

  • claude
  • neon
  • nextjs
  • trpc
Share this project:

Updates