Facial Landmark Recognition

Facial landmark recognition allows you to detect a number of different points on your face that together make up your eyes, mouth, ears, nose, and so on.

Comment

Inspiration

We were inspired by the technology of Augmented reality which has now become commonplace on several platforms such as Instagram, Snapchat, and Facebook.

What it does

Facial landmark recognition allows you to detect a number of different points on your face that together make up your eyes, mouth, ears, nose, and so on. From there you're able to apply overlays aka filters to get your snap! But there's so much more that it can be used for like emotion analysis and face tracking.

How we built it

We built it with the help of Tensorflow.JS and React. At first, we Install the Tensorflow.JS face mesh model and then build a Javascript React.JS face landmark detection app that detects landmarks on your face in real-time using your webcam.

Challenges we ran into

The first major challenge that we faced was to importing face mesh from @tensorflow-models/face-landmarks-detection, and after that, we also faced many difficulties in moving those facial landmarks detected along with the face, and we also faced difficulties in understanding the face coordinates in 3-D view. We were trying to make an app that first detects the facial landmarks through a live web camera and then after detecting those landmarks converts them into the shark face which matches those landmarks, we were trying to do this with the help of Google cloud vision API's but unfortunately, we were not succeeded on that part of converting human face into shark face.

Accomplishments that we're proud of

Although we were not succeeded completely, still we succeeded in detecting facial landmarks through a live web camera that allows one to detect a number of different points on their face that together make up their eyes, mouth, ears, nose, and so on.

What we learned

We learned mainly about react Tensorflow.js, and augmentation reality.

Built With

Share this project:

Updates