HeyEddi CI — AI Reviewer & Safe Runners

AI-powered pull-request reviews with evidence-backed suggestions and safe CI execution.

Comment 1

Inspiration

Gemini Code Assist’s GitHub review experience was being deprecated, and that exposed how dependent teams can become on automated code review living inside pull requests. We wanted to build our own alternative—one that was transparent, cost-aware, evidence-based, and fully owned by the HeyEddi product.

What it does

HeyEddi CI reviews GitHub pull requests using bounded context and AI-assisted analysis.

It detects functional changes and missing tests, identifies security and quality risks, suggests code changes and behavioral tests, and publishes evidence-linked findings directly to GitHub.

It also reacts with 👀 while processing and 🚀 when complete. Developers can use:

  • /heyeddi review
  • /heyeddi again
  • /heyeddi check
  • /heyeddi overview

Suggestions are advisory and never applied automatically.

How we built it

We built HeyEddi CI as a GitHub App backed by FastAPI, Vue, Firebase/Firestore, Google Cloud Run, Pub/Sub, Terraform, and the OpenAI API.

The webhook receives GitHub events, acknowledges the PR immediately, and sends durable review work through Pub/Sub. A deterministic triage layer runs first, then the bounded diff is sent to the configured OpenAI model. The result is validated so findings can only reference changed files before being published back to GitHub.

Challenges we ran into

The hardest challenges were making asynchronous review reliable, preventing duplicate webhook processing, preserving Terraform-managed infrastructure without overwriting Cloud Run releases, and ensuring AI output could not invent evidence outside the diff.

We also had to design around cost. Full-repository analysis is expensive, so HeyEddi uses bounded context, deterministic triage, and explicit escalation instead of sending everything to a large model.

Accomplishments that we're proud of

  • Built a working GitHub App and production webhook gateway.
  • Deployed the system on Cloud Run with Pub/Sub and Firestore.
  • Added AI-generated findings, code suggestions, and test suggestions.
  • Added immediate PR and comment reactions for visible processing status.
  • Implemented bounded repository overviews and empty eddi-ci.yaml guidance.
  • Added Terraform destroy protection and secret-managed production deployment.
  • Successfully tested the system against a real sandbox pull request.

What we learned

We learned that AI review is only useful when developers can trust its evidence. Structured output, changed-file validation, bounded context, clear status reactions, and human-reviewed suggestions matter as much as model quality.

We also learned that infrastructure and product behavior must be designed together. Retries, idempotency, secret management, cost controls, and deployment safety are all part of the user experience.

What's next for HeyEddi CI

Next we will finish model escalation from low-cost reviews to deeper analysis, build HeyEddi Runners for isolated declared CI jobs, implement Smart Affected Testing for monorepos, show complete PR history and reviewer-versus-runner costs, add maintainer authorization and stronger command controls, and expand from review suggestions to verified test execution through safe runners.

Built With

Share this project:

Updates