Imagine always knowing where the closest (and cleanest) place nearby to answer nature's call. We did. Imagine the knowledge shared by the public after a natural disaster channeled towards the resources to help them. We just faced some of nature's fury here in Florida, and many of us were without water, safe roads or shelter in the aftermath of recent hurricanes. Getting the right information about damage to infrastructure can be a challenge, and we provide a way to solve this issue by allowing people to crowdsource information to authorities.

What it does

The basic premise is simple. We allow people to tweet at a bot which collects data and then categorizes, analyzes, and visualizes this data, which is then made available to the public via a web-accessible mapping engine.

How we built it

We built two twitter bots, one which allows the public to send tweets and images of infrastructure damage, and the other dealing with toilets. information from these bots may be pre-tagged, or untagged. in the case of untagged images, we use a computer vision approach to process the images, and a machine learning approach (will be eventually a deep neural network, possibly an RNN or CNN) to categorize these images. For this project, this portion was run on an NVIDIA Jetson TX1 board which is basically an embedded board powered by a GPU. Then, the data is channeled to a mapping engine which is running uber-deck-gl, a powerful data mapping platform authored by Uber and recently released. Apart from the machine learning component, the system is run entirely on AWS. domains were bought this morning for both the damage mapper and toilet mapper, and these should be active to point to the AWS endpoints within 24 hours (registrar limitations). We discussed, and are in plans to build a document management system whereby expert or trusted users would be able to go through the repository to tweeted images, and relabel any mistakes, as well as tagging untagged images to aid the machine learning process.

Challenges we ran into

The network here has a few ports blocked, including those used for ICMP services like ping. This proved quite a headache as we have several web services running on various ports, and these need almost constant communication. Also, many issues regarding machine learning and computer vision, which are solvable on a timeframe of weeks or months, not hours.

Accomplishments that we're proud of

We were awake the whole time, and actually pulled off most things we aimed for.

What we learned

AWS services, intricacies of Node, Computer vision, cooperative programming, and finding creative ways around many problems

What's next for tweetmaps

Expand, this idea can be applied to many many different use cases. We plan to first build a large repository of labelled images which will enable the machine learning component to become more robust, and eventually utilize powerful methods like RNN's and CNN's.

Built With

Share this project: