• 

  What it looks like when being used

Inspiration

I was doing these filters for commercial reasons on Instagram and Snapchat. Most of the time I wanted to do something simple and it their approval process was too long. I then decided to create Filtrou.me.

What it does

You can create your own filter within seconds. There is an image editor where you arrange images and text and then you publish it. With the link you get after publishing you can share and when opened the filter will execute.

How I built it

I started using Jeeliz Face Filter API. But tracking was bad. And it would never track my face.

So I decided to use face-api.js to do it. But they don't provide head pose, just 2D facial points location.

So I decided to use OpenCV (compiled to WASM) to do it. But it was a 1.3mb download and it wouldn't work on iOS 12.

So I decided to train an AI to solve the PnP problem.

Challenges I ran into

3D is hard. Every program or library will use a different frame of reference (y+ goes up or y+ goes down) and it almost drove me mad.

Only when I started to draw everything on paper and really read the documentation of everything I was using I was able to make it work.

Accomplishments that I'm proud of

I trained my first AI with data that I generated. I wanted to build a product that should work on lower end phones which have really bad internet and I was able to do it.

What I learned

Draw. Think. Plan. Check your data. Plot your data. Analise your data. Code later.

What's next for Filtrou.me

I would love to see people use it. And as it's open source I hope people can learn from it. I'm also planning to upstream head pose recognition to face-api.js to make it easier for others to do stuff like this aswell.

Please, read the blog

I also wrote 2 blog posts talking about Filtrou.me. I would love if you could read it: https://filtrou.me/blog

Built With

• faceapijs
• firebase
• gatsby
• parcel
• react
• tensorflow
• three.js
• typescript