We created this application with temple students in mind. A lot of us live offcampus and safety is something thats really important to us and our friends so we created an application that would use crime data in order to keep students safe on-campus and off.

What it does

Hindsight is a web application that uses millions of lines of data taken from openstreetmap's api in order to predict the safest path students should take to their destination depending on the time of day.

How we built it

We built this application using React.js for the front end, javascript and node.js for the back-end, and firebase for our server. We initially created a bounding box for temple universities campus using openstreetmap's api and queried the for all crimes in the area. We then produced data visualizations of these crimes in order to give us more information about our data. From there we converted osm files into geoJSON for parsing and created a custom graph data structure in order for us to make custom routes for our data. These routes are rated on a weighted graph based on the amount of crime in the nearby areas.

Challenges we ran into

One of the biggest challenges we ran into was dealing with millions and millions of lines of data from the open street map api. Some of us have had experience with big data sets but nothing of this scale and capacity. One of the things we really had to look out for when implement different graph algos was our runtime complexity if our algo was more than O(n^2) it would take forever to compile. There was also a lot of data cleanup we had to do for those millions of lines in order to parse it into our custom data structures there was over 42 different types of crime in our initial data but we had to parse out for ones that were the most important to use for example assault and theft were highly weighted in our graph while gambling was not. Another thing that was a challenge was trying to create a machine learning model for our application. Another challenge was transforming and dealing matricies with numpy for our data visualization models. Learning react native was challenge for a lot of us for our front end it was a new technology but we really wanted to challenge ourselves.

Accomplishments that we're proud of

We're proud of being able to tackle and parse such a large dataset. Our initial dataset of crime in the temple area from 2015-2019 was around 2million rows long. We were able to parse through this data, create an accurate depiction of what it means, and build a tool that takes advantage of everything we learned. For a lot of this was also the first time we've worked with technologies like firebase, react.js, numpy, and geojson.

What we learned

  • how to parse millions of lines of data efficiently
  • how to apply machine learning models large datasets
  • how to use react.js for front end with a node.js backend
  • how to use firebase for application that require a lot of data
  • how to effectively view a streetmap and manage graph data with million of points

What's next for Hindsight

We hope to expand our application and make it more accurate. We didnt have a lot of time during the 12 hackathon to accurately create routes for our data and properly create a machine learning model for future crime prediction

Share this project: