Not everybody knows dancing, we trained a human-pose-human network by implement pix2pix network.
What it does
Given a dancer who knows dancing, you can generate a video that you dance in the same way!
How we built it
We downloaded Youtube dance videos and collected videos for ourselves. We first train a pix2pix network matching 2d pose extracted by openpose and RGB video on collected videos. Then dancing pose are extracted from dancer videos and apply on collected videos.
Challenges we ran into
Network implementation and training was a issue due to huge computations. Then we decided to use Google Cloud Platform TPU to train it.
What's next for YouKnowDance
We are going to put it online and train with more data to get better performance.