Repo Referee

Repo Referee uses open AI models to review code, flag potential bugs and security issues, and track token usage and costs—all through a simple red-card and yellow-card system.

Comment

Inspiration

Code reviews are an essential part of software development, but finding bugs, security risks, and potential problems can take time. AI can help developers review code faster, but its feedback needs to be understandable, and the cost of running AI models should not be a mystery.

We wanted to explore a more engaging and transparent approach to AI-powered code review. Inspired by the familiar rules of a sports referee, we created Repo Referee, where code issues receive a red card or yellow card based on their severity.

What it does

Repo Referee is an AI-powered code review tool that helps developers examine code changes before merging them.

Users can provide a GitHub pull request or paste or upload a code diff for review. The application analyzes the submitted changes and presents findings through a referee-inspired interface:

  • Red Card: Highlights serious potential issues that may need to be resolved before merging.
  • Yellow Card: Flags potential concerns that deserve further investigation.
  • Token and Cost Visibility: Displays AI usage and cost information to help developers understand the resources used during a review.
  • Model Options: Provides access to the AI model options available in the application.

Our goal is to make AI-assisted code review easier to understand while helping developers consider both code quality and the cost of AI inference.

How we built it

We built Repo Referee as a web-based developer tool with a straightforward interface for submitting code changes and viewing review results.

The project is designed to use open-weight AI models through Nebius Token Factory, with model options presented in the application, including NVIDIA Nemotron 3, Kimi, and DeepSeek.

We combined the code-review workflow with a visual severity system and a usage-and-cost scoreboard. The referee concept gives the interface a recognizable identity while helping users distinguish more serious findings from issues that need further attention.

The project also explores how open-model infrastructure can support practical AI developer tools.

Challenges we ran into

One of our main challenges was turning AI-generated code analysis into feedback that developers can understand and act on. A review tool should do more than produce warnings; it should help users understand why a change might be problematic.

Another challenge was balancing the user experience with model selection, inference usage, and cost visibility. These elements need to work together without making the interface overwhelming.

We also had to think carefully about how to communicate issue severity. A red or yellow card is easy to recognize, but the underlying finding still needs to be evaluated against the actual code and development context.

Accomplishments that we're proud of

We're proud to have turned the referee concept into a working product experience with a clear identity and a focused purpose.

We're especially proud of three ideas:

  • A memorable review experience: The red-card and yellow-card system makes code-review findings easier to scan.
  • Cost-conscious AI development: Usage and cost visibility make the economics of AI-assisted code review part of the experience.
  • A practical use case for open models: Repo Referee explores how open-model infrastructure can be applied to a tool developers may use in their everyday work.

Building Repo Referee also gave us an opportunity to connect AI infrastructure with a concrete software engineering problem.

What we learned

We learned that building an AI-powered tool is not just about connecting a model to an interface. The surrounding experience matters: how users submit information, how findings are explained, how severity is communicated, and how usage is measured.

We also learned that cost transparency can be an important part of AI product design. Developers should be able to understand the resources their workflows consume, rather than treating inference as an invisible process.

Most importantly, AI-generated findings should support human judgment—not replace testing, security validation, or responsible code review.

What's next for Repo Referee

Our next step is to make Repo Referee more reliable, evidence-driven, and useful for real development workflows.

We plan to explore:

  • More actionable findings: Include clear code evidence, file and line references, explanations, and suggested fixes where supported.
  • Improved review quality: Test against both vulnerable code samples and clean changes to evaluate useful findings and false positives.
  • Better GitHub integration: Improve the pull-request workflow and make reviewing changes more convenient.
  • Transparent model comparisons: Make token usage, latency, and cost comparisons more consistent and clearly distinguish measured results from estimates.
  • A stronger developer workflow: Explore review history, configurable severity thresholds, and integrations that fit existing development processes.

Our long-term vision is to make Repo Referee a trustworthy AI code-review companion that helps developers understand potential risks, make better-informed decisions, and use AI more transparently.

Built With

  • ai
  • code-review
  • cost
  • developer-tools
  • github
  • llm
  • nebius-token-factory
  • nvidia-nemotron
  • open-source
  • optimization
  • react
  • typescript
  • vite
Share this project:

Updates

Submission history