DevBot

Build Bot removing unused imports
Build Bot fixing syntax errors
PR Buddy refactoring code and implementing error handling
PR Buddy optimizing inefficient code
Test Bot writing unit tests

Inspiration

We are firm believers in the saying "time is money," and one major source of time spent by developers is in the pull request review process. As aspiring software engineers, we aimed to find a way for developers to spend their time more efficiently. From this, we developed a suite of developer tools we call DevBot.

Function

For context, a developer submits a pull request when pushing their changes to production, on which dozens of checks are run to ensure it is up to standard. Often, these checks fail due to simple things such as linting errors or bad tests. Upon failure, they would have to go through the console output, fix what failed locally, and then push those changes remotely.

Our first step is fixing these build failures, upon which BuildBot automatically parses the console output for what errors occurred and where. BuildBot then generates fixes for these errors and suggests the changes on the pull request for a developer to accept or reject.

Our next tool is PR Buddy, which assists developers in the review of pull requests. Upon triggering PR Buddy with the command "/prbuddy", it automatically fetches the files changed in the pull request and reviews the code, looking for areas of improvement such as refactoring, efficiency, and error handling. PR Buddy then suggests these changes and discusses why it made these decisions in a conversational manner.

Our last tool is TestBot, which automatically writes tests for code. When triggered with the "/testbot" command, it looks through all the files changed in the pull request and generates test cases for them if necessary. It then commits these new tests to the pull request branch in files labeled "{file_name}.test.{file_extension}".

Development

DevBot is written in JavaScript and bash script. It runs as a Node.js server hosted on Heroku and listens for its event triggers to automatically run its workflows.

Challenges

The biggest challenge we faced was model hallucinations. We tested models such as GPT-4o, Claude 3.5 Sonnet, and LLama3-70b. Even with highly engineered prompts, these models produced inconsistent outputs when given a prompt and a code snippet.

Accomplishments

This is our first project purely working on the backend. There were a lot of issues we had never encountered before, but we managed to push through. Consequently, we have achieved a working demo very close to what we expected coming into this hackathon.

What We Learned

We learned a lot about the pull request review process, as we had to understand every part of it to build our product. In addition, we got very familiar with CI/CD and DevOps workflows, as it is a core part of DevBot.

What's Next for DevBot

We want more compute to fine-tune and deploy our own models that are much better at programming. We also want to expand our product to include other CI platforms to enable more developers to use our product.