Interestify

Unlock the World of Research: Explore, Customize, and Connect with the Latest Scientific Breakthroughs!

Comment

Inspiration

Our project, Interestify, was inspired by the idea of creating a platform for users to easily access and explore research papers across various scientific fields. We wanted to bridge the gap between researchers and enthusiasts by providing a user-friendly interface where they could discover and engage with the latest advancements in their areas of interest.

Learning Experience

Throughout the development of our project, we gained valuable insights and knowledge in several key areas:

  1. Web Scraping: We learned how to leverage web scraping techniques to fetch research papers from the Springer website. This allowed us to provide users with a diverse collection of articles from different categories such as Physics, Chemistry, Medical Science, and more.

  2. Category Customisation: We implemented a dynamic category selection feature, enabling users to personalise their reading experience. By allowing them to choose their preferred categories, we aimed to provide a tailored and relevant content feed.

  3. Blogs, Bookmarks and Likes: The platform allows users to access blogs categorised based on their interests. They can discover the most popular blogs in the "Best" section, explore the latest articles in the "Recent" section, and find trending content in the "Hot" section, showcasing the most liked blog in the past 7 days across the category. This feature encourages users to delve into popular and engaging research papers. Moreover, users have the ability to bookmark and like articles that capture their interest, while also being able to conveniently access their browsing history and bookmarked articles in the dedicated "Bookmarks" section.

  4. Similar Articles Recommendation: Leveraging machine learning techniques, we developed a recommendation system to suggest similar articles based on the currently read blog. By offering relevant content suggestions, we aimed to enhance user engagement and facilitate further exploration.

  5. User Activity Tracking: We implemented an activity tracking system, enabling users to monitor their reading habits and track their progress over time. This feature aimed to promote accountability and provide insights into personal interests and learning patterns.

  6. Weekly Email Digests: To keep users engaged and informed, we incorporated a weekly email system. Users received personalised emails containing the top blog from their most interested category. We utilised an SMTP server for seamless delivery and a smooth user experience.

  7. Subscribe/Unsubscribe Functionality: We integrated the option for users to subscribe or unsubscribe from the mailing list, allowing them to manage their preferences and tailor their email notifications.

  8. Backend Technologies: Our project utilised the Neo4j graph database to handle complex relationships between articles, categories, and users. Firebase Auth was implemented for secure user authentication and authorisation.

How Interestify Works

During the signup process, users are asked to specify their category preferences. A "BROWSES" relationship is then established between the user and the chosen category in the database. Initially, all categories are given equal preferences, which are also stored in the database along with the relationship. When a user visits the front page, a request is sent to the backend to retrieve articles, and the articles displayed are based on the user's preferences. Categories with higher priority will have a greater number of displayed blogs. After a user reads an article, the priority of all categories is updated. The update function takes into account the decay rate of an article. If a user previously browsed a category but has recently stopped browsing it, the blogs from that category will appear less frequently in their feed.

For email services, we utilize the smtplib library and a thread scheduler to ensure that emails are sent out every week at the scheduled time. To generate similar articles, we employ scikit-learn. In addition, we are utilizing Redis as a caching mechanism for storing the cache of similar articles. This allows us to efficiently retrieve and serve similar articles based on cosine similarity calculations.

Challenges

Developing Interestify came with its fair share of challenges. Some of the notable hurdles we encountered include:

  1. Web Scraping Complexity: Implementing web scraping for fetching research papers required careful consideration of the website's structure and data extraction techniques.. We encountered challenges such as potential rate limiting and data consistency issues. To address these challenges, we implemented a timeout session that triggers the scraping method every 30 minutes. This approach helps prevent server blocking and ensures a smooth data retrieval process.

  2. Recommendation System Development: Building a machine learning-based recommendation system to suggest similar articles was a complex task. We faced challenges in optimising performance, and ensuring accurate and relevant recommendations.

  3. Data Management and Relationships: Managing the relationships between articles, categories, and users in the Neo4j graph database presented challenges in terms of query optimisation, and ensuring consistency across the platform. Since a blog website consists of thousands of blogs we had to plan our architecture accordingly.

  4. User Engagement and Retention: Encouraging users to actively participate and engage with the platform was a constant focus. We had to design intuitive user interfaces, provide seamless navigation, and continuously improve the content discovery experience to enhance user satisfaction and retention.

Despite these challenges, we persevered and overcame obstacles through collaboration, research, and iteration. We are proud to present Interestify as a comprehensive platform for exploring research papers, fostering knowledge sharing, and connecting individuals passionate about scientific advancements.

What's next for Interestify

  1. Enhanced Recommendation System: We aim to further refine our machine learning-based recommendation system. By incorporating advanced algorithms and techniques, we will provide even more accurate and personalised article suggestions, catering to the unique interests of each user.

  2. Expanded Content Sources: While we currently scrape research papers from the Springer website, we plan to expand our content sources to include other reputable platforms and journals. This will broaden the range of available articles and provide users with a more comprehensive research experience.

  3. Follow Author feature: This feature will allow users to stay connected with their favourite authors and receive notifications whenever new research papers or articles are published enabling them to personalize their research experience

Built With

Share this project:

Updates

Submission history