The reason which I have to make this project is that the original code provided by the course doesn't work and uses deprecated Python2. It was also quite an experience to have to build this project up from scratch. This is a Deep Reinforcement Learning application specifically Deep Q-Learning. Ensure that your python interpreter is 3.7 or later. There are many python libraries required to run this application: numpy, math, matplotlib, pygame, os, random, pytorch. Ensure you install the correct version of pytorch, for instance if your machine does have CUDA, then you should intall the version with CUDA to accelerate the AI process. Otherwise it will pin your CPU to death. To execute the application, just run the "" file. If your IDE is unable to run it, go to CMD, cd to that project directory, and then enter the command: python To use the Override function, click on it and you can use your keyboard's left and right arrow to control the car. Use your cursor to draw the road. If you want to erase part of your road, you must clear everything... If you give the AI enough training time, the AI will actually do quite well in the environments that you have made. To think about: Is it really a requirement to pass in 2 orientation values to the AI one is a negative of the other to the AI? Removing one of it doesn't really change anything in my opinion since both are practically the same with just a additive inverse.


This application is not perfect and there are several things that will be added and fixed in the future.

  • Do not use the Override function when you are training the AI.
  • The Cosine Rule function sometimes return a domain error.
  • Insufficient amount of data given to the car for it to optimize for the best route. Thus it will ignore the road and instead just crash onto it...
  • Sometimes suicide and crash into the sand which is attributed to finding new approach to finish each epoch.
  • The reward function is not optimized for the car to find the shortest path... Leading to the car commiting weird acts to reach its goal.
  • The buttons on screen is not reliable.
  • The erase button has not been added.

Final Thoughts

Doing a project like this would let you realize how good the libraries provided by PyTorch and Keras TensorFlow. I definitely understood how powerful these libraries are as the parameters feed to the AI are just (a lot of) numbers/Tensor. They learn pretty quickly and I believe that adding more parameters for the AI would definitely be able to let it learn even better. Algorithms like SLAM could also be used for the navigation. But that calls for a different project. You need not follow the project idea, you could implement the things learnt here to make a Flappy Bird AI that takes in the slit height for the obstacle and the bird's current height relative to the slit. I believe that this will allow the AI to learn to destroy Flappy Bird within the first 10 minutes of training. The libraries are just that good!

Built With

Share this project: