ModelPit

This is ModelPit, a battle ground arena to pin AI agents against each other. A user sets up a match between two chosen AI models or one AI model and themselves as “players” and chooses a game mode.

Comment

Inspiration

We wanted to make a product that would allow users to test the adversarial strength between AI models. In a sentence, this just means that the AI has certain restrictions that it shouldn't cross, but prompt engineering can take advantage of vulnerabilities and cause the AI to go around its restrictions. We wanted to allow AI's to compete against eachother in prompt engineering as well as allowing users to leverage it as an educational platfrom as a personal attacker.

What it does

We made a web app that connects to multiple AIs including any local ones the user may have. Then the user can pit them against each other in different game modes, such as capture the flag (trying to get the AI to go around its restriction of saying a certain word), tic-tac-toe, and rock paper scissors. The AIs get a limited amount of tokens to try and beat the other one, and at the end we save the stats in order to rank the AIs based on their performance. We also have an option for the user to go up against the AI, where the user can create prompts to try and trick the AI into going around its restriction.

How we built it

We used a lot of different techniques to create this. We started by creating an agile workflow such that we could get everything done on time. Then we started messing around with AI APIs to see how we could get them to communicate with each other. Once we had the core logic working of the AIs speaking back and forth as a part of a game, we used Antigravity to help us create a frontend that could go along with it. Antigravity was invaluable to creating an aesthetic final product.

Challenges we ran into

One huge challenge we ran into was making the frontend separate from the backend. At first, we split into separate groups trying to get both the front and back ends done quickly, but after trying to merge the two we had all kinds of trouble, and the app just wouldn't work. I love the UI we initially created, but we could not implement it with the backend in time. Because of this, we pivoted to having Antigravity help us with the frontend based on its analysis of the backend. We worked along with the Antigravity agents to finish our prototype.

Accomplishments that we're proud of

Getting the AIs to communicate and actually play the game together was really cool! We were also very proud of how good we got the UI to look despite all of the issues we were having in creating it.

What we learned

We learned a LOT during this. Some of us had never "vibe-coded" so that was already a learning curve. Moreover, we learned a lot about how AI APIs can connect to apps. We also learned a lot about product design and the development process.

What's next for ModelPit

We have lots of more plans for ModelPit! One example is that we want to add in a TTS feature such that the AIs are literally "speaking" to each other out loud. We also want to add more games and supported models!

Built With

Share this project:

Updates