Review Buddy

Say goodbye to manual pull request generation and code review hassles. Review Buddy automates the process for you. Tag @review-buddy, relax and watch the magic happen

Comment

🤖 Review Buddy 🤖

Say goodbye to manual pull request generation and code review hassles. Review Buddy automates the process for you. Tag @review-buddy, relax, and watch the magic happen! 🎉

Inspiration 💡

Developers spend roughly 30% of their time in a sprint maintaining existing code by working on mundane tasks such as addressing code review comments and fixing bugs with stack traces from sentry/kibana/pagerduty, etc. We have personally seen the pain of working on such tasks and wanted to boost our productivity by leveraging AI tools so we can focus on more important problems of building great products and solving hard problems.

What it does 🛠️

Review Buddy seamlessly integrates with existing developer tools like GitHub/Asana/Jira/Sentry/PagerDuty to extract useful context from these tools and autogenerate code reviews.

How we built it 👷‍♀️

  • Build integrations with various data sources like GitHub Actions/Asana and PagerDuty webhooks to fetch relevant context for the coding task.
  • Indexed GitHub repos using vector embeddings in Pinecone.
  • Use embedding-based retrieval for providing context to GPT.
  • Apply the changes smartly on GitHub.

Challenges we ran into 🚧

  • Training LLMs to return data in a structured format.
  • Lower token limits for complex coding tasks which require context of several thousands of lines of code in a repo.
  • State management for version control.
  • Latency optimization, especially when chaining several calls to LLMs.

Accomplishments that we're proud of 🏆

  • Ease of using Review Buddy in our day-to-day work.
  • Picking up knowledge of the LLM stack in a matter of 2 days with no prior experience building LLM apps.

What we learned 📚

  • Prompt engineering needs better tooling and frameworks. Clearly a nascent space that's changing rapidly.
  • We need ways to evaluate the quality of LLM apps (accuracy/precision) as there is an explosion of tools. Especially when it's being used for tasks like pushing code to production that could impact real users.

What's next for Review Buddy 🔮

  • Better optimizations in the model to use LLM-based retrieval instead of embedding-based retrieval.
  • Use models with larger token limits such as Anthropic.
  • More tooling integration to make Review Buddy the co-pilot for reviews.

Built With

Share this project:

Updates