Self Driving Car with reinforcement learning & CV

Inspiration

There is a Science Family Night at my our school and each person is going to make a cool demonstration of a science phenomenon or a piece of technology and explain how it works to the lower grade students. We thought machine learning is the biggest thing right now so we decide to make a self driving car our self.

What it does

It uses Cozmo robot as a car, the program reads each frame from the camera of the robot and feeds it though two neural networks that we built ourselves. The neural network makes the decision of the steering of the car.

How we built it

We first built a driving simulation so we have a controlled environment for training the neural network. We uses an autoencoder to reduce the dimension of the environmental state from 80 vectors to 2 vectors. Afterwards, we used a Deep Q learning algorithm and another neural network to create the state reward function. When we train the network, the robots starts off steering randomly. Every time the car goes off road, we give a negative reward to the agent network. By correcting the agent network several times, we were able to train the agent network in a very short time.

Challenges we ran into

The part I spent most time on is when loading the neural network model, Tensorflow library does work well with the Cozmo Robot sdk. This cost me almost 2 hours to find and fix the error. Also, the autoencoder

Accomplishments that we're proud of

We are able to train the neural network in a very short period of time. The autoencoder is able to eliminate many environmental noise from the camera frame.

What we learned

Robot is harder to code because their behave differently each run caused by the inconsistencies in the environmental factors like lighting and people walking around the room.

What's next for Self Driving Car with reinforcement learning & CV

Use a DDPG network to get continues output of steering angles rather than choosing to only turn left or right Having the machine learning model to learn to stop when it encounters a obstacle.

Built With

• cozmo-robot
• cv2
• keras
• pil
• python
• tensorflow