A basic architecture of how FaceMark works under the hood.

Vote for FaceMark on LinkedIn!

Inspiration

The inspiration for this project started when as soon the Covid-19 pandemic started, we had to stop using Bio metric attendance systems in our university. The government of India makes it compulsory for all universities in India to track attendance, and as most colleges use Fingerprints to do so, we were faced with a new dilemma - how to perform the same scale of "people recognition" in classrooms where more than 200 students sit together - It was no longer possible for them to touch the same machine again and again and increase the risk of infection.

And hence was born FaceMark - A completely new, non-intrusive way to mark people present by simply placing an edge computing device in each room with a camera attached to it.

What it does

FaceMark, at its core, is a way to simply recognise the people in the room. Many previous solutions do that, but what makes FaceMark stand out is that it solves the problem no other solution in the market does - it scales extremely well, and due to it's perfect implementation of RedisEdge.

While building this, we had to keep in mind that most universities, like ours - hold multiple classes (upto 25) at once in the same building. Currently no single system could scale to handle and process the live feed coming in from these rooms while keeping the costs low, until now!

How I built it

The project entirely relies on the wonders of Redis. To ensure that everything, from image capturing to processing for facial detection, to recognising the faces uses the very fact that Redis relies on in-memory storage. Our previous implementation included saving all images and files to disk and then picks them up to process them.

Using RedisEdge (A stack compromised of Redis, RedisAI, RedisGears and RedisTimeSeries Modules), we were able to take the video feed into Redis, quickly process the frames using RedisGears, store the frames in Redis and then find out the faces using the AI module, store that in a stream and pass it off to another RedisAI module to calculate and recognise the face. All this data will be passed off to RedisTimeSeries which will store metadata about people in the frames rather than storing it in a different database. This will be particularly useful for analyses of the data later, and to use that data to build the output frames if needed.

This same process earlier involved having each module run by a cronjob, and dumping it's output to disk, and the next module picking it up from there, which led to a high latency. Now, this process can be done in almost real-time!!

Future implementations actually also include using RediSearch to make our video feeds searchable for the presence of a person.

Challenges I ran into

I faced issues because I had never worked with Redis before. But the biggest challenge I ran into was working with multiple RedisGears at once. RedisGears run in memory, which makes them incredibly hard to debug (You have to dump the execution stack to trace an error). Additionally, I didn’t find much documentation on initializing multiple RedisGears inside the same Redis server, and had to reach the working conclusion by hit and miss. Apart from that, Redis is a pure bliss to work with, and the documentation covers literally all possible API methods the libraries have!

Accomplishments that I'm proud of

Making this work! Woohoo! I have been working on the research side of this project for a couple of months, but never really had the idea of how we'll make it production ready to sell it to potential customers. Now we do! This demo demonstrates exactly how we can do it and make it a proper product, and there's nothing more I'm proud of than to have been able to make it!

What I learned

I learned a lot of Redis - from the different tutorials online and the resources. Even more, I learned about real-time processing and that Redis has so much to offer!

What's next for FaceMark

FaceMark's journey has just started. After finishing the prototype, we will be testing it on the data we have collected and trying it out live as soon as colleges and universities start opening. From there, our feedback cycle will begin which will surely lead to a better and more finished product!