YOLO Brawlers

Two Robots Fighting Full View
Pose Detection Demo with YOLOv11
Side View of Final Arena and Microcontrollers
Single Robot Full View
Over the Shoulder Fighting Shot
Fighting Shot Close Up
Top View of Final Arena

Inspiration

We Love Robots! We wanted to build a project that we could consider a truly fun endeavour while delving into the most exciting avenues of robotics, where leading companies are pushing the boundaries of humanoid and companion robots. These innovators are developing robots capable of performing a wide range of tasks while enhancing human-robot interaction through advanced AI, neural networks, and deep learning. The growing focus on creating robots that can seamlessly integrate into daily life inspired us to develop a more accessible and affordable option. Our goal is to provide individuals with the opportunity to explore the potential of humanoid robots without the barriers of high cost or complexity. By offering an approachable yet fun entry point with robo-fighters, we aim to help bring the future of robotics to everyone.

Check out our demos:

Demo 1
Demo 2 (This one has a better fight scene!)

What it does

Using the latest pose detection models, our tabletop robots connect to your laptop's camera feed and translate your movements from human player to robot fighter instantly.

How we built it

We repurposed existing toys similar to Rock'em Sock'em models by reassembling them and attaching custom servo motors controlled by ESP32 microcontrollers to enable movement. We combined the power of YOLOv11's pose detection with the microcontrollers to control the fighting robots. The client-side Python code analyzes your moves through OpenCV, while custom C++ firmware controls the servo motors that bring the robots to life. A PyQt user interface lets you easily connect your laptop to your mechanical fighter, making setup a breeze.

Challenges we ran into

Perhaps our biggest challenge was the hardware itself. Building a responsive fighting robot meant making tough decisions about motor placement and assembly. We went through multiple iterations to find the sweet spot between mobility and stability, ensuring our robots could throw a mean punch while staying upright!
Optimizing pose detection for real-time response - because no one wants their robot throwing a punch five seconds late to the party.
Building a rock-solid WiFi communication system between laptops and robots - dropped connections mid-fight? Not on our watch!
Configuring the backend development environment for running OpenCV and the YOLO V11 Pose Estimation model on different operating systems

Accomplishments that we're proud of

We implemented a fully responsive robot fighting system architecture that actually works in real time! The smooth translation from human movement to robot action using computer vision feels incredibly satisfying. Plus, we managed to keep the cost and complexity low enough that anyone can join in on the robot-fighting fun.

What we learned

This project taught us the ins and outs of real-time computer vision, the delicate art of servo motor control, mechanical constraints, microcontroller programming, and the challenges of wireless communication in robotics. We also learned that making robotics accessible doesn't mean sacrificing sophistication - sometimes the most elegant solution is also the most fun!

What's next for YOLO Brawlers

We're gearing up to add more fighting moves and unlock walking movements, letting our robots dance around the arena! We're also working on packaging our user-friendly app, so you can jump straight into the action without any setup headaches. Plus, we're planning to make our design open-source, encouraging anyone to build and customize their own robotic gladiator. The robot revolution starts on your desktop!

Built With

c++
esp32
opencv
pyqt
python
yolo

Submitted to

HackED 2025
- Winner First Place

Created by

I developed C++ embedded software for ESP32 microcontrollers that enables WiFi connectivity. This socket infrastructure allows real-time control of a robot's punching and weaving movements through a PyQt desktop application. I also worked on breadboarding the circuit that would convert the old hydraulic fighting mechanism to a servo-motor controlled system, and integrated the ESP32 microcontrollers for precise joint manipulation. Contributed to the translation system that converts YOLO pose detection outputs into robot movements, bridging the Python and C++ components of the project.

Jared Drueco
5th-Year SWE @UAlberta
I developed the Computer Vision classification method by processing the YOLOv11n model's keypoint output data into classifying different fighting poses using OpenCV and Python.

I also worked on the client-side Python code for sending the determined fighting poses to the ESP32 microcontroller over sockets

Finally, I worked on developing the PyQt application, particularly on components like selecting a player, choosing between Keyboard or Pose Controllers, etc for a seamless experience in playing the game

I contributed slightly to the initial C++ firmware code for controlling the motors and building a few parts of one of the robots

Sami Jagirdar
Year 5 Software Engineering @ UAlberta
Worked on developing the PyQt UI, integrating the OpenCV camera in the UI, utilizing the YOLO 11n pose model to capture pose frames from camera, building blue robot, and documenting the development process in the README.md.

Aron Gu
5th Year Software Engineering Student @ UAlberta
Worked on integrating the yolov11n model into our workflow. Then transitioned building both robots and then modifying them to be controllable using our ESP32 microcontrollers. Also worked on setting up the UI for the start screen as well as designing the pixel art for the whole project. Recorded the demo video and took many of the promotional shots for the project. Worked on the integration of all hardware components into the final product including stage building, robot servicing, and rewiring.

Antonio Martin-O.
I am fourth year software engineering student from the University of Alberta. I am interested in learning about robotics.