• 

  The tool you needed before the hackathon begun

RepoRoast: The AI-Powered Code Debate Panel

Inspiration

In an era of vibe coding and rapid prototyping, the traditional bottleneck for developers isn't just writing code—it's validating it. We noticed that manual peer reviews are often biased, slow, or overlooked entirely in hackathons and startup environments. We wanted to build an agentic panel that doesn't just scan for syntax errors, but debates the soul and viability of a codebase. RepoRoast was inspired by the need for an unbiased, multi-perspective architectural audit that developers can access instantly.

How we built it

RepoRoast is built on a high-performance, real-time streaming architecture designed to provide feedback in seconds, not hours:

• Frontend: Built with Next.js 14 and TailwindCSS. We used Framer Motion for the debate animations and Recharts to visualize the multi-dimensional scoring of the repository.
• Backend: A robust FastAPI server manages the agentic workflow.
• Agent Orchestration: We utilized LangGraph to create a panel of three specialized judges:
  • Alex (The Innovator): Focuses on novel patterns and future-proofing.
  • Sam (The Architect): Focuses on modularity, coupling, and production-readiness.
  • Jordan (The Impact Assessor): Focuses on user accessibility and real-world utility.
• Data Pipeline: The backend parses GitHub repositories, chunks code via LangChain, and uses ChromaDB for vector storage to retrieve relevant context for each judge's specific domain.

Our scoring logic aggregates the agents' feedback using a weighted confidence interval, calculated as:

$$ S = \sum_{i=1}^{n} (w_i \cdot c_i) + \delta $$

Where $S$ is the final audit score, $w_i$ represents the weight assigned to each agent's evaluation category, $c_i$ is the agent's confidence score, and $\delta$ represents the system's dynamic volatility factor based on codebase age and dependency depth.

Challenges we faced

The biggest challenge was orchestrating a multi-agent system that could stream content in real-time. Implementing Server-Sent Events (SSE) in a way that kept the UI responsive while the backend agents were still thinking was non-trivial.

We also faced "Agentic Hallucination" where judges would sometimes agree with each other too quickly. We had to implement strict system prompts and a conflict-resolution node in our LangGraph to ensure the debate stayed critical and constructive rather than just echoing the repo's README.

What we learned

We learned that agent orchestration is significantly more complex than simple LLM prompting. Managing the state between agents (the "debate") requires a rigorous graph-based approach. We also learned the importance of UX feedback loops: by streaming the agent responses in real-time (typing animations), we dramatically reduced the "perceived latency" for the user, turning a slow audit process into an engaging, theater-like experience.

Next Steps

We plan to integrate automated PR comments, allowing RepoRoast to automatically leave review comments directly on a user's GitHub Pull Request, closing the loop between audit and implementation.

Built With

• fastapi
• langgraph
• next.js
• tailwind