I want to create a complete web application which uses MEAN stack, Spark and Scala. My aim is to process a data set and do some filtering with Spark and store the results on MongodB and finally use the data and display it on the front end.
Hence I have taken the Food inspections data set from San Francisco site. The data contains the business name and the associated risks. The risk levels are high, medium and low.
What it does
My spark job reads the input CSV data contains food inspection details and processes it and stores the data in MongoDB as collections. I have allFoodInspection and filterFoodInspection collections here. The first one has all the data and the second one has the business name, the unique risk category and number of risk’s committed.
My MEAN stack REST layer reads the data from Mongodb and processes and exposes the data and the Web Layer uses the data and display it and use the data for drawing a chart.
How I built it
I have used Spark, Scala, MongoDB, NodeJs and AngularJs to do this.
Challenges I ran into
Accomplishments that I'm proud of
What I learned
Spark, Scala, MongoDB, NodeJs and AngularJs
What's next for San Francisco Food Inspection
Need to add lots of details and chart with the data