Mario Kart ML: AutoDrift

• 

  This corresponds to line 514 in the table

• 

  Data table for the inputs

Inspiration

Mario Kart

What it does

Mario Kart: AutoDrift is an application that utilizes machine learning to learn how to play Mario Kart 64.

How we built it

It is built around Python, C, Keras, Tensorflow and other API's and libraries. We used an emulator to allow us to have inputs directly into the computer that would control how Mario is driving on the screen. We then captured the inputs of the controller and the images of what was occurring on the screen simultaneously from actually playing the game. We then fed this data into python to model it correctly so we could feed it into Tensorflow to train the model. Finally after we have the trained model, we unleash it in all its glory using a python script to beat a level in Mario Kart!

Challenges we ran into

• Writing the Tensorflow Model
• Getting the integration correct between TensorFlow and the mupen64 emulator.
• Reading the data in correctly from the player input and images into a file
• Deciding on a game!
• Compatibility between API's

Accomplishments that we're proud of

• Used Tensorflow to train a model
• Learned Tensorflow from scratch
• Gathered inputs from an emulator and wrote them to readable data by Tensorflow
• Fresh installed 2 Operating Systems on two systems to improve integration.
• Got to enjoy a nice time playing some old school video games

What we learned

We recognized that the learning curve for Machine Learning was as difficult as to be expected. Tensorflow was very moody when it came to trying to get it to work the way we wanted it to. However, we were successful in implementing and training a Keras-wrapped TensorFlow model and stumbled at the last minute in integrating the results back into the emulator. With an extra hour we could have a real-time demonstration of our AI's learned capabilities. In addition to this we learned that a project all of the team is passionate about is worth more than just trying to make a project that only one or two people enjoy.

What's next for Mario Kart AutoDrift

What's next might not come explicitly from Mario Kart; we learned that we can extend this to many other video games if desired, such as the timeless Super Mario Brothers, Sonic the Hedgehog and to go all out, possibly Super Mario 64. We can also apply this to the real world and the rapidly changing environment for self-driving cars! These algorithms could be adapted to be implemented in your next self-driving Uber!

Built With

• keras
• mupen64
• scikit
• tensorflow