CAMEL Judge Workforce

Best Project in the hackathon

Comment

Inspiration

Multi-agent system is designed for complex tasks. In this project, we want to use this system to mimic a real-world judge committee.

What it does

Each judge in the committee is an AI agent, and they are all evaluating a hackathon project. Each judge will have a unique identity and persona, and they will give a score and a comment to the project according to the criteria they are evaluating.

The judges are:

  • Visionary Veronica: a venture capitalist who is obsessed with how projects can be scaled into "unicorn" companies.
  • Critical John: an experienced engineer and a perfectionist.
  • Innovator Iris: a well-known AI startup founder who is always looking for the "next big thing" in AI.
  • Friendly Frankie: a contributor to the CAMEL-AI project and is always excited to see how people are using it. The different personas offer diversity and the judge committee as a whole will give a comprehensive evaluation of the project. Finally, the project will parse the feedback and give a final summary of the opinions. The final result will be parsed into a JSON format with OpenAI beta API and output to a data folder. In this project, we also implemented a front end web page to retrieve the data from the data folder and put the results into a leaderboard in real-time.

How we built it

We used the Workforce system in CAMEL-AI as the framework to build the multi-agent system.

Challenges we ran into

The tasks may lose details when decomposing tasks. To solve this, we added a new field in the task design, which contains important issues, and this information will be preserved with best effort to make sure the important details are delivered to each agent.

Accomplishments that we're proud of

Our committee produces robust and reasonable comments, while still being funny and preserved each judge's unique personality. We also have a front end page that looks fancy.

What's next for CAMEL Judge Workforce

We will make the communication model more flexible.

Built With

Share this project:

Updates