We designed Falcon with an all-too-familiar scenario in mind: remembering snippets of content or meaning from websites but not the exact title or URL. Unfortunately, modern browser histories are only indexed by these two properties. To this end, we designed Falcon, a multi-platform application that allows users to search their browsing history by page content and the semantic meaning of the embedded images.
As the user browses the web on Chrome or their smartphone, Falcon keeps track of the pages they visit, and extracts both text and image content. We index the text in an elasticsearch database, and feed the images through a deep recurrent neural network to infer and articulate what is going on in the image or gif. We use a server written in golang to index and search all the data received from users.
A Chrome extension and an Android app feed the data to the server, and users can search through a native popup-style Mac client or an omnibar extension for Chrome.