TrueJet Reviews

Aspect Based Opinion Mining of Social Media and Review Websites With Natural Language Processing for Airlines

Comment

Inspiration

We both love to travel and wanted to make it more accessible to others. JetBlue is striving to make travel more enjoyable and available to everyone and we wanted to contribute to their mission.

What it does

We webscrape airline opinion data from review and social media websites and then use natural language processing to determine the subject and polarity of review. The data is then graphed by category and percent satisfaction, and compared airline to airline. Our goal was to evaluate JetBlue’s customers’ thoughts throughout the flying experience. Understanding the customer’s opinion will allow us to better inform JetBlue on how they can increase customer satisfaction.

How we built it

Our data is web scraped from review sites, because of their reliable verified opinion, and Twitter because the average customer is more likely to post on Twitter as opposed to a review site. Once the data was web scraped, we processed the reviews sentence by sentence to figure out aspect category and polarity using opinion mining. Opinion mining was used because we could tag a sentence with multiple categories and label specific terms. The flexibility to define polarity of words and meaning of the sentence improved our model’s accuracy. Next, we parsed sentences for key words and sorted them into categories and used a series of natural language processing libraries to gather sentiments and tie them to categories. To process our scraped data, we trained a natural language processing neural network. In the end, a series of generated graphs that relate common flying experiences to a positive or negative sentiment informed our recommendation. We graphed what experiences were most common to review and what percentage of people liked that aspect of JetBlue. To see where JetBlue can improve, we compared it to its airline peers. We even established our own metric for evaluating airlines using a custom “customer satisfaction” index.

Challenges we ran into

Annotating tons of reviews for aspect category and polarity and learning natural language processing from scratch

Accomplishments that we're proud of

We are proud of how much we accomplished in a short amount of time and the skills learned. We are super excited about the graphs and all of the great data we could gather. We are also proud of how accurate we were able to make our neural net, 91%, in such a short time.

What we learned

We learned all about web scraping and working with large data sets as well as natural language processing!

What's next for TrueJet Reviews

We want to create live updates to see what reviews of airlines happen everyday and expand our sources of review data. We also hope to create more detailed review categories to better understand users' needs.

Share this project:

Updates