Deltadoop

Inspiration

Wanted to deploy my skills with big data in a realtime project. (Sagar Thapar)

What it does

It gives a correlation between the sentiment related to delta and the weather and WSA times fetched from their API. It can give the company a visualization about how customers are reacting to certain situations, so that they can analyze their performance.

How I built it

I used Apache Spark streaming on top of Microsoft Azure HDInsight and Power BI for data visualization.

Challenges I ran into

Setting up the Spark Cluster

Fetching data from the API

Accomplishments that I'm proud of

Making something that works in real time.

What I learned

How to use microsoft azure.

Using two agents or clients to fetch data from different sources.

What's next for Deltadoop

I would like to add some more variables and machine learning to the whole experience, so that the analysis becomes more intuitive and we can move towards a more predictive phase.

Built With

• azure
• hdinsight
• particle
• power-bi
• spark-streaming
• twitter