gif of a few iterations
I wanted to gain some experience with Tensorflow and I found an interesting paper on genetic optimization of neural networks.
What it does
I found a cool challenge in the open AI gym, which provides environments for testing reinforcement learning systems. I tried to apply the algorithm outlined in the paper to this problem but was largely unsuccessful.
How I built it
Final Algorithm works in several steps: 1) Generate 500 neural networks with completely random weighting and bias 2) Simulate all of them, keep the 10 highest-performing specimens 3) Make 5 copies of each specimen and make minor mutations to them 4) Goto 2, repeat as many generations as desired
Challenges I ran into
Tensorflow isn't friendly with unusual topologies of networks so I had to give up on half the algorithm and instead try to optimize just weights and biases in a fixed-shape network.
It's really hard to combine neural networks (genetic crossover).
The optimization process found a lot of local maximums, so it would hone in on a pretty weak solution.
I'm not sure if the neural network I built is even capable of learning to walk.
Accomplishments that I'm proud of
I was able to make artificial neural networks control a figure and attempt to walk. There was some level of genetic optimization, so I guess it kind of worked?
What I learned
Basic Tensorflow syntax, studied reinforcement learning and genetic optimization algorithms.
What's next for Evolutionary Neural Networks for Walking Biped
While my experiments were largely unsuccessful today, I would still like to complete this OpenAI challenge. I think that the evolutionary optimization of ANNs is either unsuitable / not practical for this application so I should probably look into a different strategy.