Scholar Scraper

Result page landing
result page hover
result page expand

Inspiration

Our mission was to help college students with their educational career in some form. Sorting through thousands of articles and sources can turn research into a long and tedious process. Our application was built to overcome this in order to make research more enjoyable.

What it does

Scholar Scraper is an application that assists college students in the research process by generating sources from Google Scholar based upon their thesis. Users can input their thesis statements and Scholar Scraper will generate a list of sources they can use. By cross-referencing keywords from the user's thesis and the abstract portion of the source, Scholar Scraper will rank the sources given based upon relevancy.

How we built it

Front End

We used HTML, CSS, and JavaScript along with React and Tailwind to create the structure and overall look of the application.

Back End

We used C#, .NET, NVIDIA API, Gemini API, Selenium, OpenRouter API, and the AngleSharp library to pull keywords from the thesis, generate an abstract, cross reference it with sources from Google Scholar, and create a list of ranked sources based on relevancy.

Challenges we ran into

Finding reliable databases to webscrape
Interacting and managing external API requests
Integrating React and Tailwind into Front End

Accomplishments that we're proud of

Our usage of the NVIDIA open router API
Making the frontend and backend successfully interact for the generation of sources.
Creating a cohesive style that was present throughout the entire application

What we learned

Through participating in HackUTD, we were not only able to compete in our first hackathon, but also gain the experience of working with new technologies, connecting with new people, and putting our skills to the test.

What's next for Scholar Scraper

We are planning on implementing features to further increase efficiency for users. In the future, Scholar Scraper will be able to search sites other than Google Scholar for sources as well as pulling information from sources in format other than PDF. Scholar Scraper will also factor the publishing date into the relevancy score to further increase applicability.