AGENTS

A project that blows the minds of the judges

Comment

Inspiration

We wanted to build a ruthless, elite "Staff Engineer" AI that judges code objectively. In a world of polite, overly-forgiving AI assistants, we needed a "brutal" benchmark for developers that punishes bad practices like tight coupling, memory leaks, and N+1 queries. We wanted to gamify code quality.

What it does

BrutalBench authenticates a developer via GitHub, dynamically pulls their latest repositories and commit diffs, sanitizes the payloads to optimize context windows, and uses a highly-tuned Google Gemini pipeline to generate a ruthless 0-100 score along with a strict, 3-sentence critical evaluation of their architecture.

How we built it

We utilized a monolithic Next.js App Router architecture for a seamless full-stack experience.

  • Backend/Auth: We used Supabase (PostgreSQL) to securely handle GitHub OAuth and persist the evaluation scores.
  • AI Pipeline: The core engine leverages the @google/generative-ai SDK, enforcing Strict JSON Schema Generation to guarantee structured, predictable outputs.
  • Frontend: Built with TailwindCSS v4 for a stark, brutalist terminal aesthetic, and Zustand for decoupled, reactive pipeline state management. ## Challenges we ran into Our biggest hurdle was dealing with unpredictable AI API rate limits (HTTP 429) and high-demand service outages (HTTP 503) on experimental models. If the primary model went down, the entire pipeline would stall. We solved this by engineering a High-Availability Multi-Model Fallback Array. If gemini-2.5-flash is rate-limited, the system instantly catches the error and seamlessly rotates to a fallback tier (gemini-flash-latest, gemini-2.0-flash, gemini-2.5-pro) with exponential backoff. We also built a hardened fail-safe mechanism that mathematically guarantees an evaluation response even in catastrophic blackout events. ## Accomplishments that we're proud of We are incredibly proud of the resilience of the AI pipeline. Moving beyond a simple "fetch and pray" AI integration, we built a production-ready, fault-tolerant evaluation engine. We're also proud of the payload sanitization logic that strips out SVG bloat and Base64 strings to drastically save on token usage before passing diffs to the LLM. ## What we learned We learned how to force LLMs to output guaranteed, strict JSON schemas. We deepened our understanding of secure, server-side OAuth token interception with NextAuth and Supabase Service Roles, and we learned how to architect robust retry/fallback loops for highly-available LLM integrations. ## Built with
  • Framework: Next.js (App Router, Turbopack), React
  • Language: TypeScript
  • Styling: Tailwind CSS v4
  • State Management: Zustand
  • Authentication: NextAuth.js (GitHub Provider)
  • Database: Supabase (PostgreSQL)
  • APIs: Google Gemini API (Multi-model: 2.5-flash, flash-latest, 2.0-flash, 2.5-pro), GitHub API
  • Utilities: js-tiktoken (for strict context-window token optimization) ## What's next for BrutalBench
  • CI/CD Integration: Packaging BrutalBench as a GitHub Action that automatically blocks Pull Requests if the AI Staff Engineer gives the code a score below 70.
  • Deeper Integrations: Expanding support beyond GitHub to GitLab and Bitbucket.
  • Granular Metrics: Adding specific scoring for Cyclomatic Complexity, test coverage, and security vulnerabilities alongside the main score.

Built With

Share this project:

Updates