cognitive search: The search result is based on the actual content of the image not the file name.
With an ever increasing number of mail accounts, online storage services , personal computing devices, a common user feels overwhelmed with the sheer quantity of files. So it is quite understandable to expect the user to have a central search/content pulling app at his beck and call. The app must be his personal service agent capable of pulling files/documents from across all the users mail & storage accounts. It should also be able to recognize other devices used by the user and fetch files from them as well. More like a Go-getter who never fails you !!!
What it does
Go-getter is a multi device / platform search application, which takes a search string from the user and searches for that file in his mail accounts (Gmail) , online storage accounts (GoogleDrive), and desktop systems (multiple PCs connected within the same network belonging to the user).
Whats more.. this guys is a real Go-getter. Using Cognitive service provided by Microsoft Azure, the app fetches images that are contextually related to the search query. It also performs cognitive text processing on mail contents and returns mails that are more relevant to the given search query.
How we built it
Three beasts worked tirelessly for 24 hours to make this work. We first came up with a high level design with a Master-Slave architecture that can do a search on multiple systems connected to the same VPC of the user. This was then extended to include content pulling from Gmail and file search/access from Google drive. The final icing on the cake was the addition of Cognitive abilities for the app, using Image Captioning and Text Analytics provided by Microsoft Azure
Challenges we ran into
Time was one of the biggest challenger but also the best motivator. Typical Gmail content is quite huge for the indexing hence the app heavily depends on Azure, rather than API pulling. Security, Discovery of nodes, cognitive search, meaningful information retrieval were few other notable challenges.
Accomplishments that we're proud of
It is an end-to-end working model with multiple functionalities. The design discussions we had were pretty intense and taught us a lot. We did a lot of this which were out of our comfort zone which we enjoyed a lot.
The end product that we built is a beast in growing.
What we learned
How to focus on high-level designing before getting on to the implementation details. How to keep a cool head when everything stops working. Out of the box thinking to provide the user with the best service possible. Here's a list of things to bullet that we learnt: -> Distributed Storage and Retrieval -> Smart indexing with Azure's Cognitive text processing -> Semantic based retrieval of multimedia with the help of Azure's Vision API. -> An introduction to Peer to Peer Protocols -> Built a platform which can easily plug, high performance data sources, with just a few lines of code, living up to it's name of future centric plug and play
What's next for Go-Getter
He is a Go-getter.. he has big dreams. Future scope includes intelligent and fast text processing using NLP. Cognitive capabilities of the app is not restricted to just mails and images, it will be extended to context retrieval across all file formats. Support will be extended to Hotmail, OneDrive , DropBox and any other custom file storage services ( that opens API support). Potential migration to cloud in cases where indexies grow large for any single device to store.