Stories come out all the time about global events and conflicts, and we're used to hearing about the major ones -- right now, probably the one we'd all think of is the war in Syria. However these are not the only events happening now, and we may be unaware of even what is happening in Syria or these other major places. Some small events that might pop up on the news might also be part of a decades-long conflict that we had no idea was going on.
What it does
Built with d3.js, world events in a certain time period can be visualized on an interactive globe, and each event contains a link to a Wikipedia article about the event as well as links to articles of larger events which a single smaller event is a part of.
How I built it
Scraping data of Wikipedia through their conflict category pages (for example, 2002 conflicts) was done through Python and the data is stored in a MySQL database. A Node.js server queries the database and serves JSON, and d3.js in the browser visualizes the data.
Challenges I ran into
Scraping data from Wikipedia was quite difficult because the page layouts are inconsistent, and layouts that are readable to humans can be very hard to read through traversing the DOM.
Accomplishments that I'm proud of
Getting a substantial amount of conflict data from Wikipedia, as well as creating a program to get conflict data from any time period was challenging and the data is not perfect, but it is very useful for this project.
Creating a world-wide visualization through an interactive globe was also difficult, to get the scale, projection, coordinates, etc. right while making it at least somewhat compelling to use.
What I learned
Getting data through Wikipedia is difficult without the use of an API, it would be ideal to use an API like this for conflict data, at a scale as finely-grained as Wikipedia's articles, instead of scraping HTML pages.
Also when using D3 to visualize JSON data, using a SQL database is probably not the best choice due to the extra overhead in converting between database rows and complicated JSON, and a NoSQL database would probably have been easier to use.
What's next for Conflicts in context
More conflicts, and more context!
Log in or sign up for Devpost to join the conversation.