Pokémon Showdown Team Stealer

• 

  poster

Inspiration

Calling all Showdown players! Have you ever wanted to just hop on and casually hit up the ladder but have been too lazy to build a team? Maybe it has been a while since you last played and you don’t know what the meta for your favourite tier is like anymore. Introducing the Showdown Team Stealer, a (soon to be) Chrome extension to help you steal teams from Showdown Live videos from YouTube! If you have ever stolen teams before, you know exactly what I am talking about. This app simply automates that very tedious process for you. To anyone who doesn’t, allow me to explain...

The Situation

Anyone who has played Showdown before knows, but the modern Showdown UI gives lots of information about the Pokemon used by either side. For now, we will be focusing on the player’s side. The most prominent way to see information about the player’s Pokemon is from hovering like so:

The Idea

We will be using this hover information to determine the stats to invest in for each Pokemon. To do this, we will have to segment each frame and identify if a hover is present. We can create and label a custom dataset and train a Yolov8n model to identify the different segments of a showdown battle frame like so:

Next Steps

After segmenting, we can split the segments and then pass them individually into an OCR model. From there we can simply calculate the EV spread and nature we would need to give to a particular Pokemon to reach the stats we see in the hover.

Challenges

• There were definitely a lot of challenges that were faced with this project. In fact, AI was not even going to be used for this project when the project idea was first conceived. It was only added on as a necessity later on.
• The first layer of AI which was attempted to be placed was the OCR one. However, after a long time of struggling, it was decided that the Pokemon Showdown battle UI is too complex for most OCR models to be able to read and organise the text. This is why segmentation was considered next.
• In a Pokemon Showdown battle, there are three types of hover elements: the player's Pokemon hover (the one we want/care about), move hover and enemy Pokemon hover. It was a bit of a challenge to get the model to differentiate between the two types of Pokemon hover sometimes and more training data had to be added to get it to a comfortable level of accuracy.
• Having no experience working with AI in the past, but deciding to use Rust from the get-go definitely made things very complicated. What could have been completed in 3 lines of Python was instead spent with weeks of struggling. Two AI libraries in Rust were tried and abandoned due to them not supporting the format of the model.

What we learned

Basically anything and everything I know about AI development now comes from this project. It was my first time labeling a dataset and training a model.

Future

Right now, OCR is done with MiniCPM-V 2.6 gguf. This model is easy to use with Ollama but may be too heavy for the purposes of this app. It may be preferable to replace it with a more lightweight model like PaddleOCR. The plan is for this app to be a browser extension. The battle chat has been included in the labeling to estimate enemy Pokemon stats from damage dealt/received. Screenshots from the Showdown teambuilder will be included in future datasets to also extract team information from there too.

Built With

• rust