These days, the Netherlands is a pretty safe place to live. If something still ruins your day, it is not going to be the end of the world. It might be the rain getting you wet to the skin, a stranger at a bar spilling their drink all over your shiny shoes, or the wind being too cruel to your umbrella. The worst case: someone steals your precious rusty bike.
On a serious note, bike theft is rampant in the Netherlands. As dictated by the Pareto principle (80/20!), to further improve public safety by eliminating petty crime would require an enormous investment of public funds... unless we bring technology to the table.
What it does
Using face recognition and object tracking, we detect suspicious events — such as a stranger trying to take your bike — in the video stream. Upon detection, our system immediately notifies you with details, such as a snapshot of the incident, to let you, your neighbours, or the police take action.
Trusted users provide images of their faces to the app. The video processing backend tracks locations of bicycles and people. If someone unknown takes the bike that is left by a trusted user, the owner gets a notification. The app can also send a message to owner's neighbors, asking them to check that things are OK.
Apart from the bicycle theft scenario, this approach is easily adaptable to other security-related contexts. Potential applications include:
- Detecting physical access control violations on secure premises, such as warehouses and offices (or even your house, while you're away);
- Strengthening access control and security screening in demanding environments: customs, border crossing points, airports;
- Preventing retail store theft.
How we built it
Our solution is comprised of three parts:
- An Android app, which allows trusted users to register their faces and receive notifications;
- A TensorFlow backend, which monitors the video stream and keeps track of bicycles and people;
- A lightweight web server, which enables smooth interaction between the backend and the app, and also dispatches external notifications using Twilio.
Challenges we ran into
Our stack is quite complex and diverse: an Android app, a web server, and a TensorFlow backend all juggling with data back and forth. We have devoted quite some of the work to making these parts smoothly work as a whole. Moreover, none of us have ever built an Android app before! It took a while to figure things out on the phone side.
Accomplishments that we're proud of
"A computer eye that never sleeps to keep your home safer" still sounds somewhat futuristic. We made it work!
What we learned
It's always worth to keep things simple. Even in a hackathon environment, where things like maintainability and test coverage don't mean the world, it might be tricky to prevent a complex system from falling apart.
First of all, we'd comb up our solution a bit – it's quite hacky. After that, it's worth to enable more flexible configuration, to support cases other than bicycle watch out of the box. Another important aspect is the legal status of our tool -- CCTV law is different around the world, so we need to be sure that we're not committing a crime while trying to prevent crime.