Too many restaurants, so little time. Yelp is great, but it's sometimes hard to get a specific grasp of really how great a restaurant is with just a 5-star rating system. We believe that people's true feelings are revealed by their words, and Big Mouth Scraper will allows us to analyze just that.
What it does
Big Mouth Scraper is a web scraper that scrapes yelp pages for a list of their reviews. We are then able feed these reviews to IBM Bluemix's Alchemy API to retrieve a sentiment score based on the wording of that review. By adjusting the rating against the sentiment analysis score we're able to obtain an establishment's "true" rating, which we can then display on a graph. Within the UI, the user is able to change the search query to target general or specific businesses at a certain location. In addition, they are able to change the number of businesses scraped, and the number of reviews per business displayed.
How we built it
The entire application: the scraper, the API request, and the UI was written in R. R swag.
Challenges we ran into
We didn't really know how to use R at the beginning of the hackathon, so this was a huge challenge for us. We understood that this was making the project unnecessarily difficult, but the challenge made the overall project a lot more rewarding. Although Yelp did provide an API for us to work with, we found it a bit fussy, and so decided to simply retrieve the reviews ourselves via scraping.
Accomplishments that we're proud of
We managed to finish this project, and overall, it looks pretty nice!
What we learned
By building this project, we definitely learned a lot about the language and how well it fits in with data manipulation tasks.
What's next for Big Mouth Scraper
We're looking for even more meaningful ways to visualize this data, and hope to incorporate other factors (reviews or otherwise) to better predict a Yelp business's true value.