We were inspired by our personal interests in the intersection of behavioral economics and the power of artificial intelligence in analyzing emotion in text.

What it does

Lotus is a web app that provides public sentiment analytics about a company using web scraping and machine learning algorithms. A user can input any company name and the app will return graphical and qualitative information about the overall public approval of the company based on real-life, internet-based data. It is intended for use both by companies (such as JetBlue) who wish to analyze their own public image, as well as the general customer interested in business analytics.

How we built it

Lotus is a multi-component program, consisting of a python-based web crawler developed entirely in-house, an ml-powered data analysis and visualization program created with AI-based Natural Language ToolKit, Pandas, Matplotlib, Seaborn, and information from the Twitter API; and finally a html/css/js-powered frontend utilizing Jquery, Ajax and the Flask microframework to run the web crawler and ml algos live from the web app.

Challenges we ran into

Our initial challenges were in data mining and web crawling, particularly in knowing what information to extract from what websites/social media outlets, as well as how to clean the data using NLP techniques (e.g., lemmatizing all words into simplest form, assigning sentiment indices to both individual words and large paragraphs using ML, etc.). Our next challenges were in using Flask to enable any user of our web app to run our back end scripts live, as well as displaying the graphical and textual outputs of our backend scripts in a consistent and beautiful format. Integrating all of our individually-developed programs into one cohesive service was a challenging but incredibly rewarding experience.

Accomplishments that we're proud of

Our web crawler, NLP sentiment analysis functions, and linguistics-based data cleaning techniques are all highly algorithmically sophisticated and developed primarily from scratch; our website is aesthetically beautiful and easy-to-use, despite the sophistication of the programs behind it.

What we learned

We learned about how to interface natural language processing, data science, web development, linguistics, and behavioral science to create a product that will be useful and informative to companies and to a wider audience.

What's next for will continue to improve its machine learning algorithms, as well as expand its data analytics to include such features as geographic specificity (i.e., provide information about where a company is doing best or worst), live and animated site analytics, and the extension of its usage toward political and social queries as well as business ones.

Share this project: