We dreamed big at the beginning to try and tackle machine learning and a much more advanced AI to play the game. However, that route didn't exactly pan out.
What it does
We have five bots built on multiple Amazon Web Services provided servers. These bots communicate with a program on the computer actually playing and streaming the game Cuphead. Also on this streaming computer is another bot, the "Master" if you will, which controls some of the IO of the other bots. All of this communication happens over the twitch chat of the game stream which happens to be IRC. The program running on the streaming computer handles the IO from the twitch chat into the game itself.
How I built it
We used python to make bots that communicate with each other. We also used python for the main program handling the IO from the chat into the game. We used the program OBS to stream all of this to twitch. The Master bot on the streaming computer takes some input from the game screen, using pyautogui, to handle different events and change the control sets of each bot to fit the situation that the game presents.
Challenges I ran into
Amazon Web Services were difficult to figure out mainly because we could not connect to them after a certain point, leaving us blind as to whether we could get the files for the bots, modify, or even turn them on at all. Figuring out how to get the input from the Twitch chat into the actual game was difficult. We tried various different methods, all seeming to not work until we tried one for a second time and it somehow worked.
What I learned
We learned that AI is difficult. On a more serious note, we learned that even if we have a great idea for something, it may not be practical to get it finished in a timely manner. We had a great idea to use both AIs and compare various statistics between them as they both played the game. This, sadly, did not happen. We learned that even if we cant complete this big idea that we dream up, it doesn't mean that we cant make a cool project that is a little less involved but still of the same main idea.
What's next for CupHead Twitch Bot AI
There are many more things that could be added to the AI. It would probably be good if the AI could learn, add a little machine learning in there. More practically, we could add more functions to the bots to make them better able to assess their situation and react accordingly. We could also add the ability to scale the entire operation by automating the creation of bots and servers. It could even be able to scale itself according to the amount of people in the chat or to adapt to the speed/intensity of the game at that moment. There are many possible improvements for the AI which could be implemented in the future.