We all lose things. It's part of being human. However, we are steadily moving toward an age where we are no longer required to rely on our own faulty recall and recognition. Already the internet and cell phones have moved us into a state wherein digitally-assisted memory is the norm. With Remember we want to move that digitally-assisted memory into the physical space.
What it does
Remember is an Amazon Alexa Skill that works in conjunction with webcams in your home to help you identify and locate your misplaced items.
How we built it
Remember is written in node.js and go. The Alexa skill is written in node.js. And, running off a raspberry pi with webcams connected to it is a go script that causes the device to act as a local server for processing the surfaces visible to the webcams. Alexa sends a signal to the pi to take pictures of the surrounding area. This triggers the go script that takes the pictures, saves them if items have moved significantly, splices them into smaller pictures to increase fidelity, sends them to clarifai for item recognition, processes the returned JSON and stores the processed results in Firebase. The Alexa skill then gets the results from firebase and tells you where the item you're looking for is located, provided that it was within Remember's vision.
Challenges we ran into
Hours upon hours of frustration and tears.
Accomplishments that we're proud of
- Taking remote photos with our raspberry pi interface
- Writing our server code to interface with multiple apis
- Using Alexa to communicate to users the information gathered about the room by our hardware interface
- Making it work!
What we learned
- How to create Amazon Alexa skills
- Using a raspberry pi as a piece of dedicated hardware with webcams for taking, storing, uploading, and processing images
- Working with Firebase
- Working with clarifai's api
- Lots of cool miscellaneous tech knowledge
What's next for Remember
Our original plan was to integrate Remember with the nest cam, but we were unavailable to acquire the hardware at Spartahack. With the nest cam, Remember would be able to take a 360° view of a room and process every visible surface for possible item locations. The increased quality of the photos, as well as the composite image processing, would make Remember better than you at finding your things.
Furthermore, we'd like to increase accuracy and usability of the Skill, through additions to the functionality available to users of Alexa, with additions to supported phrases and abilities.
Development of a mobile app and web interface to view stats about where your things are most often, and suggest possible locations for lost items. System tracks locations where items are lost and uses machine learning to help determine where you most often leave your items if the system can't detect them with its connected camera/s. This would be further integrated into the Alexa skill so users could ask Remember where it thinks their items could be.