We wanted to create a platform which can take data asynchronously and analyze it. Once the analysis is complete, the trends are shown on a web app and updated in real-time.
What it does
Our project accepts incoming news headlines and associates each headline with a category. Over time, the system keeps count and popularity of each topic.
How we built it
We used Node.js and Python for different parts of the system. To add data and simulate a data stream, we used Python to insert data into our MongoDB database. We used Node.js + Express + Angular for the web application part which shows the visualization.
Challenges we ran into
We initially attempted to use Docker for different nodes in our system but ultimately there were technical difficulties which convinced us to avoid Docker for now.
What's next for RealTrends
We need to remove the effect of articles after a fixed time to avoid having count values growing forever.