Minerva

Output data after the bot runs

Inspiration

Looking around for inspiration, we realized that there weren't any course catalog sites that aggregated all the vital statistics for a given course as well as incorporating ratemyprofessors.com data.

What it does

The project is a fully-automated data-scraping routine which outputs its data in a clean, easy-to-read web format. Essentially, the user types in the course they're looking for, and the product automates all the work of aggregating data (like course sections, meeting times, professor and professor ratings, etc.), saves it in a .CSV file on the user's machine, and outputs it in a series of visual cards on our website.

How we built it

Using a UiPath flowchart, we created a series of steps that would form the control flow for our project. We used UiPath's data-scraping tools to collect course information from online sources, as well as to collect professor quality and difficulty ratings from ratemyprofessors.com. We then processed the data within UiPath and wrote it to a .CSV file. With React.js, we built and hosted a website that would facilitate graphical output of our data, with an emphasis on readability. Finally, we automated the process of uploading our CSV file to the website, eliminating the last bit of human interaction from the project.

Challenges we ran into

Getting the data scraping algorithms to work right was tougher than expected. Our UX design proved difficult to implement.

Accomplishments that we're proud of

Our project is 100% automated -- after the initial user input, the entire process of gathering, saving and outputting data is completely free of human intervention. We're also proud of getting the ratemyprofessors integration running, as that's our project's unique selling point.

What we learned

The possibilities of automation are endless. Any process doable by a human is also automatable (if you've got a lot of time.) UiPath is really cool.

What's next for Minerva

Our ultimate goal is to get the app running server-side and fully integrate it within a web-app interface. This will reduce the friction between the user and the program. We'd also like to improve our UX design, especially our course data view, and increase the scope of our data-gathering.