Main Dashboard UI to enter user interests as well as configure them (adding and deleting it)
Sign up and Log in page for user to register and enter
Mail inbox with system generated mails
UI for displaying previously sent recommendation and a feedback system to incorporate User Ratings
System generated Email for the recommended article sent to user

Inspiration

We are B.E. (Bachelors in Engineering) students and wanted to learn about new architecture in Machine Learning. For this we had to read new research paper in the same field but when we tried to do so, the websites were confusing and cluttered at the same time. It was difficult to surf through those websites to get a good research paper especially for an undergrad.

This is when we thought that recommendation for research paper is a good project idea, which can help new people interested in reading research paper with an easy entry.

What it does

It is a research paper recommendation service, with an intuitive UI.

It delivers what it promises, that is weekly mails with recommended papers or articles according to your interest configuration using SPRINGER API.

How we built it

It is built using MERN stack and python. When user interacts with our Web UI, it sends the required data like signup time, user configured interest and user email to our backend.

Our backend handles the user registration and authentication using JWT tokens and MongoDB. When the user configures his/her interests, this emits an event in Node and creates a child process running python script.

The python script then uses SPRINGER API to fetch all the articles springer has to offer related to the user interest. This list of articles are then sent to recommendation system.

Our recommendation system uses TFIDF encoder to encode every articles’ abstract and title. Then similar articles and interests are grouped together. This is again cross referenced with the user ratings of past recommendation.

Finally, a score is prepared to serve most recommended article. This is saved in DB, which is then used by mail scheduler to send emails at regular interval by checking the time of last mail in DB.

We did this completely remote, we scheduled regular meets for writing and testing the following code.

Challenges we ran into

There were few challenges like:

Handling event loops in node became very complicated with the increase of API endpoints, for instance asynchronous calls
Initially we faced many restriction, due to policies of multiple platforms like Yahoo and Gmail. Scheduling emails through SMTP were tough due to blocking and non-blocking code running simultaneously.
preprocessing fetched data from the SPRINGER API was a challenging task due to nested JSON and text formatting
using WordNet to search through synonyms of user interest was difficult since, we had to run few installers while spawning the child process
integrating feedback system in front-end, back-end and database, we had to come up with new scoring system which incorporates the user ratings

Accomplishments that we're proud of

We were successful in designing a scalable, easy to implement and efficient system
Email scheduling which runs smoothly was the most complicated part which we tackled through persistence
We learnt how to work in a team (remotely) and brainstorm together as a team
How to properly showcase demo of a working project with it's intricacies and all the documentation
Consistently keeping track of project by committing it on GitHub regularly

What we learned

All these difficulties helped us at learning about:

event loops in node and how to handle them
child process spawning and detaching it with parent process to run it asynchronously
scheduling mails, using blocking code like setInterval and setTimeout with non blocking code which is used to fetch database.
TFIDF encoding of textual data using sci-kit learn and using cosine similarity to find similar text groups in embedding space.
Resolving database conflicts, like VERSION ERRORS and overwriting issues by different processes.

We also learnt how basic things which are never thought of can be simplified with simple software.

What's next for Journal-Hive

adding OAuth authentication for Gmail API integration
integrating multiple languages into the same platform
making a recommendation article for platforms like Medium
making a ranking system for QnA platforms like Quora and social media platforms like Twitter
making a YouTube video recommendation by encoding its caption transcripts