Have you ever heard of a club from a friend, and though "Wow, that's really cool! I wish I had known about it earlier". Or been intimidated by the huge, cluttered bulletin boards where people post endless announcements for events that may or may not be over already. Finding a club or doing something fun to do takes a lot of time even on a college campus, but we don't think it should be like that. We built Surf to eliminate the gap between students seeking fun student orgs and the clubs which don't have the presence to advertise and grow organically.
What it does
Problem: students don't have time to go searching for orgs, and most clubs don't have time to go out recruiting. We solve this by web-crawling primary CMU websites, email-crawling a honeypot email that is subscribed to all the student mailing lists, and delivering them to students in one nifty searchable database. It's even exportable as a Google Calendar, and can be filtered through to find events fitting your interests, say "Sports" or "Food". With a streamlined interface, Surf eliminates all of the frictional cost of looking for things to do or clubs to join and increases students' willingness to engage with their communities.
How I built it
We built the email crawler using Google APIs and the web crawling in pure python. The NLP to identify events by their keywords in text used Microsoft Azure's Keyword identifiers. Finally, Surf's website was built in Django, allowing execution of our python scripts.
Challenges I ran into
We had a hard time working with web crawling because HTML is finicky, even when you have BeautifulSoup. Not to mention, CMU's advanced websites were largely dynamic, making it difficult to recursively travel down links where most links haven't loaded. Google's email API was well-documented, but still difficult to learn to use for the first time.
Accomplishments that I'm proud of
We divvied up the work very well, and each of us got to experience a new technology. One member learned Django in a day, another worked with APIs for the first time. All in all, a wonderful learning experience making something useful and fun with your friends.
What's next for Surf
We believe Surf really has the potential to connect students and orgs in a way that is frictionless and easy for both parties. In the future, we want to improve our event finding algorithm to achieve higher consistency, and parse websites in a cleaner way that allows capturing dynamic features. We also want to implement a recommender system for events that learns what the user likes. And besides, who can turn down club meetings with free pizza?