Bruinwalk scraper and analyser

Inspiration

I am a college student at UCLA and I've found that how well I do in my classes depends greatly on how suited the professor's teaching style is to my learning style. Unfortunately, the official university website for students to give reviews of professors at UCLA is hard to navigate and makes choosing classes finding out what I want to know about classes difficult.

What it does

For a given review of a class, the program scrapes through all pages of reviews and returns the number of occurences of a each keyword specified in the Python program. Any number of keywords can be specified.

How we built it

It was a simple script built in Python using the Selenium and BeautifulSoup4 packages.

Challenges we ran into

This was my first actual Python project, my first personal project (outside of school and class), and also my first hackathon.

I ran into difficult with setting up my Python environment (learning about how to use virtual environments and install packages) and also how to manage dependencies (separating base and dev dependencies).

Accomplishments that we're proud of

I managed to learn about how to set up my Python environment and manage dependencies all on my own. I have never done this before.

What we learned

Learnt more Python syntax, how to use Selenium, learnt to use pip-tools, and learnt to create demo videos and project descriptions for hackathons.

What's next for Bruinwalk scraper and analyser

I have a plan for the next Hackathon I'm participating in to use the program I built for this hackathon to build a larger program that returns keyword-related data on all the classes and all the past offerings of the class by different professors that are related to a specific department (for example, Computer Science). I would compile the data on common keywords related to entire departments and store it in a database.

Hopefully this is a tool I can share with other students at my school once completed.

Built With

• python
• selenium