Update 1: More stable face generator!
We've been figuring out how to make Juntos a more expressive and also realistic face generator. Primarily, we've fixed two main problems:
- The generation of unrealistic, oversaturated faces
- Changing an attribute like nose size also changes age or gender
After consulting with the ML community at Caltech and digging deeper into recent papers, we've converged on the following solutions:
- Seeding the generator's latent vector with a random starting image
- Orthogonalizing the "age" and "gender" axis with all the other attributes
Here's a brief explanation of why it works:
- The problem with our demo previously was that our latent vector, which generates an image, was initialized as 512 zeros. This causes any perturbations to create huge relative differences between vector values (basically, dividing by zero). By seeding the vector with a 512-dimensional standard normal, we are able to have much more stable generation.
- Naturally, our GLM finds like attributes like "hairline" are heavily correlated with "age". By orthogonalizing, we remove the "age" part of "hairline" vector to create a purer "hairline" attribute that you can manipulate without changing the rest of the image.
Next steps
We hope to pursue facial reconstruction next, so you can modify images that you can upload! The greatest barrier to that is creating a cycle-consistent face encoder, which will require some sort of auto-encoder or bi-GAN. To be continued!
Log in or sign up for Devpost to join the conversation.