One time I was trying to teach myself about the Karnak Temple, and I was having trouble finding the right resources to learn what I needed to. I could never think of the right keywords to use to find what I needed. I found one good page, but after that, I didn't know where to look. So I thought what if there was a tool to search for more content like what I had found, kind of like a reverse image search from Google, it would make my life a whole lot easier.
What it does
StudyAdvisor aggregates data on user preferences as they browse educational internet resources. Built on a strong foundation formed by collecting data from across the internet, StudyAdvisor identifies correlations between pages based off of the concepts discussed on the different pages. These correlations are fine-tuned in reaction to the actions of users to optimize your research experience with a collection of crowdsourced data and automated tools.
How we built it
We first built a web crawler to automate searches to find top pages distinct subject areas. We then scraped these pages to distill the fundamental concepts discussed therein using the IBM Alchemy API. We proceeded to cross reference the sites with each other to find correlations based on the subjects they discussed. We built a browser extension that improved our dataset by recording users' searching habits, allowing us to provide the most relevant recommendations based on what our users are researching.
Challenges we ran into
Our first problem we ran into was setting up the linode server to run on ubuntu 14.5. After quickly solving that with the help of linode sponsors we ran immediately into another problem with IBM's alchemy api key becoming expired. We made a decisive decision to be much more accurate with our next two keys to make sure we had a functional database. Our last feat to overcome was our Dijkstra algorithm to search for close related concepts to current users sessions.
Accomplishments that we're proud of
We're proud to have made a resource that can help users world wide find online resources. We are proud to have made a database of over 9000 correlation mappings. Our little child has grown to explore the internet from one little linode.
What we learned
We learned how to implement different API's and implement our own api's through flask. Organizing our large mySQL database was an interesting learning experience as well. Finding correlation between these was a great learning experience in terms.