I had a project which implements this algorithm with caffe2. I am moving to tensorflow recently and want to use the facial landmark more often with tensorflow 2.0. Therefore, I spent sometime to reimplement with project with tensorflow in python.
What it does
The project implements 2DFAN4 which was proposed in paper "How far are we from solving the 2D & 3D face alignment problem?". It landmarks 68 facial keypoints with a glasshour network.
How I built it
I rewrote the code which was written in c++ in python and modeled the network with tf.keras interfaces. The training was monitored with the tensorboard showing the loss and the facial landmark result on a test picture.
Challenges I ran into
The challenge I met is how to manage so many layers with tf.keras interfaces. All layers need to be initialized before they are composed into a whole network. Then I figure out that if I decompose the network into more smaller modules and construct the bigger network with the smaller modules, the model is so simple that it can be described in around 100 lines python code.
Accomplishments that I'm proud of
The training is successful. I get the desired parameters in only one try. The implement of 2DFAN4 with TF2.0 in python enables me to use the facial landmarker without building the supporting framework libraries such as opencv and caffe2 from source. The python libraries can be used out of box.
What I learned
I learned how to model the network with tf.keras interface and how to monitor the training with tf.summary
What's next for 2DFAN4 facial landmarker
I am going to implement the deep prediction part of the 3d facial landmarker with TF2.0.