Inspiration - We found that LLMs do not always produce the most accurate output when given a techical question. After asking the LLM to edit and make changes several times, only then did it generate an accurate answer.
What it does - CRAFT involves 5 agents, coder, reviewer, auditer, fixer, and tester. Each of these agents work together to analyze and edit the code to make sure it produces an output that is up to standard. If not, it will loop and keep reviewing the code until it passes a certain benchmark, and passes all the tests.
How we built it - We used python and VScode to write this project with some assistance from LLMs. Our process included building each individual file from CRAFT and synthesizing them all together in a main.py file.
Challenges we ran into - The two largest challenges we faced were dealing with API issues and limits, and the tests not resetting and regenerating. We combatted the API issues by researching how to limit the token and using them sparingly as we did not have unlimited. The testing issues was fixed by ensuring they would reset and be rebuilt each time a new prompt was entered.
Accomplishments that we're proud of - We are proud of creating a working successful agentic loop that can recursively self-improve and self-correct.
What we learned - We learned many new skills and gained new knowledge such as what and how to build and apply agents in the real world, and how agents can shape how AI is used in the future.
What's next for CRAFT — Code, Review, Audit, Fix, Test - We plan to improve the algorithm and optimize the speed and accuracy even further, making it more resource efficient with tokens, and also linking it with GitHub repositories to edit and improve all the code in a project.
Log in or sign up for Devpost to join the conversation.