CodeGoat

Inspiration

The project is inspired by CodeRabbit. CodeRabbit is a code review platform that integrates with GitHub and knows your repository inside out. The internal structure of how CodeRabbit does its analysis of the repo is something that inspired me to build this project.

What it does

The main feature of this project is that it creates briefs for the whole repository on initial analysis using a Recursive Language Model and gets accurate briefs responding to user queries and doing code reviews. You can do code reviews for your PR or ask the agent to do test case coverage. Ask it anything about the repository, and it will answer.

How we built it

The app was built with the help of Codex and GPT-5.6. I started out by creating the appropriate initialization pipeline to get enough metadata for the state graph. The initialization step accumulates all the details about the repo and the PR and loads it into the state of the langgraph for the agent and the tool nodes.

The main feature of the application is a Recursive Language Model that generates briefs for the files in a repo. It is a programmatic RLM function that goes through the files and generates file-level, directory-level, and repository-level briefs. These briefs provide the knowledge base for the agent, which it can access by querying a Supabase Table.

After that, users can chat with an agent that has access to multiple tools to fetch briefs from the Supabase DB, get the PR diffs, and file names.

The agent decides which flow or tool node it needs to trigger based on the user's query and responds accordingly.

Challenges we ran into

I ran into several challenges when building the knowledge base for the agent. For instance, the vector DB I intended to make would not have worked in this case, as the code snippets are hard to analyze without a bigger picture. So, I decided to go with an RLM function that generates a brief file-by-file, as it is the perfect architecture. The repo is already a worktree, and all the function has to do is generate a brief and also condense its findings for each brief into directory-level and repository-level. This helped prevent the context rot problem, which would have caused an agent to lose memory for the previous files as the briefs still persisted in the recursion function.

Then the challenge was building the agent itself. The problem was how to change the state and what should be in the state. The files could have caused the model's context window to fill up quickly. That is why I came up with a workflow for looking at a diff, fetching the right briefs, and restarting the process for the next one. This way, only the relevant context gets loaded into the agent's context window.

Then there were problems with deployment for the application. There were some CORS issues, and I resolved them with Codex by connecting the Render plugin so it can access my blueprint.

The tool nodes were another problem that I faced. I didn't want to give full access to the agent for choosing the parameters in the tool, as the user has given complete Git access and not just a single repo. Therefore, I loaded some scopes into the state for the tool nodes to get the parameters from scoping the results from the DB and Composio toolkit for git to the specific repo and pr user is working on.

Accomplishments that we're proud of

I am proud of the RLM function I built, as it is a nice example of how it can be used. The RLM function is the highlight for me, as it helped me build a lightweight knowledge base for the agent to use.

The StateGraph I built is another thing I am proud of because the result turned out to be good and the responses are highly relevant to the PR and provide nice test case coverage.

I have been testing the application for nearly three days and so far, I have only used $1.04 worth of credits for the testing phase which includes the RLM function.

What we learned

I learned how to build RLMs and StateGraphs from scratch. I also learned how to use Codex, and the annotation feature helped me build a nice frontend for the application. I primarily use Cursor for development, but the results from Codex turned out to be even better.

I also learned a lot about CodeRabbit's internal architecture. Although CodeGoat is not nearly close to it, I believe that it reduces the token usage for repo analysis as compared to CodeRabbit.

What's next for CodeGoat

I plan on improving the briefs generated by CodeGoat as they are simple .md files. I plan to add graph dependencies to make the results more relevant and fetch highly relevant files from the knowledge base.

I also plan on changing the knowledge base from Supabase to something that is much faster and provides better partitioning. Right now, all the briefs exist in one table for all the repos, and the only thing that fetches the relevant content is the chat scope in the state graph that filters the briefs. I want better partitioning so users cannot tamper with the state and access other users' repo briefs.

I also want to increase the speed of the RLM function, as it takes a long time to generate briefs. Moreover, it can break if the user refreshes or decides to start another RLM loop. Handling multiple users comes next.

Then come guardrails for the agent. I have made it so it doesn't have write access to the file. I would prefer to make a CLI tool next to be able to write files locally for testing or making bug fixes.

Built With

• composio
• github
• langgraph
• postgresql
• react+vite
• supabase
• typescript