Mar.AI

Our AI is trained in Time-Trial mode, but can still race in Grand Prix mode.
A demonstration of the search process. The AI simulates the outcomes of 11 angles, choosing the angle yielding the greatest progress.
The paths that the kart takes using the DAGGER approach (Green Lines). The blue line shows the path of the CNN AI playing in real-time.
A visualization of the first layer activation functions for a single input image.

Inspiration

What it does

Mar.AI is an AI in Mario Kart 64 for the BizHawk emulator. Essentially the game is run using the bizhawk emulator and a python server with the model is started which interacts with the emulator through a lua script.

How I built it

This was built using Keras with a Tensorflow backend due to time constraints using NVidia's CNN Architecture. I then used weatherton's lua scripts to read some the the input from the memory rather than determining where all the values are stored. Afterwards, it was simply a matter of connecting the systems (done through a local server) and training the AI.

Challenges I ran into

The biggest challenge I ran into was configuring the entire ecosystem. The neural network model was built, however I still had to determine what types of input and output I would be able to use. I decided on image input which could be read from the game memory rather than the computer screen so that the size of the window did not matter. Then to simplify the game, only steering output was determined which made the process a lot easier.

Accomplishments that I'm proud of

I am proud of creating an RL algorithm which was sped up using the DAGGER search algorithm to determine a decent solution which was then imporved through the use of a CNN

What I learned

I learned a lot more about Imitation Learning and how using that process in addition to a CNN, would allow me to actually improve the system alot. Rather than simply calculating all the possible Q-values from the map, or simply estimating it, I calculated a smaller portion, learned, predicted, and kept iterating.

References

TensorKart - The first MarioKart deep learning project, which was used for inspiration.

Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning - The idea for using a search-based AI for teaching the Convnet AI came from this paper.

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning - The DAGGER algorithm was first introduced in this paper.

MarioKart 64 NEAT - This AI uses the NEAT algorithm to genetically evolve a shallow neural network

weatherton/BizHawkMarioKart64 - Some MarioKart 64 scripts which we used as a reference for memory read/write locations