Inspiration

Ever since I was a young child, I've always wanted to be Linus of Linus Tech Tips. I wore socks and sandals every day.

Today, we've made that into a, uh, reality.

What it does

It takes a picture, uses deep learning to crop faces out of it, uses more deep learning to convert the face into Linus' face, and pastes it back on.

How we built it

The interesting part of it uses what's called a variational autoencoder. The autoencoder is comprised of two pieces, an encoder and a decoder. The encoder takes the image and lowers its dimensionality -- it loses information -- into what's called the latent vector. The latent vector is then fed into the decoder, which is trained to reconstruct the image.

By doing this, the encoder learns to learn the "most important" parts of the image and place that in the latent vector.

So, for example, the encoder could place the position of just one eye, since the other eye could be inferred by the position of the first.

By doing this it is possible to use the encoder on not-Linus' face and reconstruct it in the manner of Linus' face but in the original face's pose.

Challenges we ran into

No one except AWS lets you have GPUs for free tier. A bummer, and AWS wouldn't accept our request to up the GPU limit for the stronger GPUs. So it was trained on a laptop with a 1050.

Servers are hard. Much time was spent on servers.

Picking hyper parameters for deep learning is an exercise for mystics.

Accomplishments that we're proud of

The technical complexity and execution. While the final product is... not terribly useful, the internals represent some of the cutting edge in the field, especially with GANs taking the spotlight.

What we learned

If you think something will be "easy" regarding server admin stuff, think again.

What's next for Hello, today on Linus Tech Tips...

Better training! Alpha layers! Real time video conversion!

Share this project:

Updates