Inspiration
My friends often beat me in shooter games, and I want to finally put an end to my shooter game ineptitude.
What it does
It sends inputs over HID to simulate mouse movements, to track enemies on your screen.
How we built it
The project is split up into two parts. The STM32 microcontroller handles the HID inputs to simulate human looking mouse movement to a final position. On the computer runs an executable which captures pictures around your mouse, finds enemy targets, and sends the STM32 a command which describes a position it should move the mouse to.
Challenges we ran into
Originally I wanted to hold all logic on the STM32, but because it only has 20 KB of RAM and a pretty weak processor, I realized that it would be unrealistic to have good target tracking run on it. Also, because a USB device can't see the screen without a program on the computer sending it the pixels, I would have to run software on the computer regardless. It was pretty difficult to track objects that move in complex patterns. I think we got pretty close to something good, but there is still a lot of work to be done on the tracking software.
Accomplishments that we're proud of
I am proud that we were able to get the system to run and work with tracking something other than a red ball. It was very fun to play around with this system against actual players.
What we learned
I learned a lot about how USB devices like mice work, and how to make a microcontroller act like one. I also learned that tracking something on screen is a lot harder than just finding it, and how to use a neural net to find players.
What's next for stm32-tracker
I want to make the tracking better for targets that move in complex patterns, and test it in more games. I also want to add more storage to the board, so the program can live on the board itself and you can just plug it in.
