Evolutionary Neural Networks for Walking Biped

GIF
gif of a few iterations

Inspiration

I wanted to gain some experience with Tensorflow and I found an interesting paper on genetic optimization of neural networks.

http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf

What it does

I found a cool challenge in the open AI gym, which provides environments for testing reinforcement learning systems. I tried to apply the algorithm outlined in the paper to this problem but was largely unsuccessful.

https://gym.openai.com/envs/BipedalWalker-v2/

How I built it

Final Algorithm works in several steps: 1) Generate 500 neural networks with completely random weighting and bias 2) Simulate all of them, keep the 10 highest-performing specimens 3) Make 5 copies of each specimen and make minor mutations to them 4) Goto 2, repeat as many generations as desired

Challenges I ran into

Tensorflow isn't friendly with unusual topologies of networks so I had to give up on half the algorithm and instead try to optimize just weights and biases in a fixed-shape network.

It's really hard to combine neural networks (genetic crossover).

The optimization process found a lot of local maximums, so it would hone in on a pretty weak solution.

I'm not sure if the neural network I built is even capable of learning to walk.

Accomplishments that I'm proud of

I was able to make artificial neural networks control a figure and attempt to walk. There was some level of genetic optimization, so I guess it kind of worked?

What I learned

Basic Tensorflow syntax, studied reinforcement learning and genetic optimization algorithms.

What's next for Evolutionary Neural Networks for Walking Biped

While my experiments were largely unsuccessful today, I would still like to complete this OpenAI challenge. I think that the evolutionary optimization of ANNs is either unsuitable / not practical for this application so I should probably look into a different strategy.