Frustrated when you can’t find your stuff? How many times have you wished you could just “Control-F” your phone or wallet?
What it does
CtrlFEverything is our web-app that helps you find where your things are in reality at any time.
How we built it
Here’s how it works: a user can set up cameras in places like different rooms at home, where a Qualcomm Dragonboard takes pictures and communicates with these cameras every minute. The pictures are sent to an I/O server, then to a computing server, which integrates Google’s Cloud Vision API. All of our hosting, databases, and vision code are backed with Google Cloud Platform. One noteworthy thing is that the I/O server does not send any pictures already processed for processing again. The processed pictures, tagged with things found in them, are sent to our database through the I/O server. Now what happens on the user’s side is, as he searches for something by either typing it or using our dope voice recognition, the I/O server grabs the corresponding pictures and present them to the user, together with the time they were taken, which devices took them, and controller IP.
Challenges we ran into
- Duplicate object labelling from Google Cloud Vision API
- Interfacing multiple cameras
- Voice recognition on the front-end
- Interfacing Google Cloud Platform to get actual running environment
Accomplishments that we're proud of
- Successfully implementing our system entirely on the cloud
- Achieving a working system of image process and transfer from the camera, to our back-end servers, back to the front-end on another computer
What we learned
- Using Google Cloud Vision API to detect object labels, and using the data in Node.js to interact with the rest of our system
- Using the DragonBoard 410c to interface over the internet, over a local network, and to take photos
What's next for CtrlFEverything
First, we plan to provide customization of device names, so you can name your cameras “kitchen”, “bedroom2”, etc. We also will provide geolocation based on the controller’s IP, so that those who have cameras in multiple locations don’t get confused. Another important feature we want to implement is user authentication: so that users can only access pictures their cameras take, or someone else’s that are shared to them. This leads to considerations of scalability, which is easy to implement on our side; all we need to do is add more servers, and perhaps databases, where we will distribute users’ pictures in different databases without confounding them.