This Pandemic has taught has it doesn't take time for a small event to happen at a major scale. If we could get alerted even at a very beginning stage, there's a lot of hope. People can stay alert and avoid such things. This apply not only to pandemic but also to small floods, tornadoes, tsunami's and even crime etc. So, we came up with a plan to use the social media platform to alert its user, based on increasing disaster/crimes for any location. #Hack_for_Good
What it does
TWEELP is an AI based SaaS web data integration (WDI) platform which send SMS/emails alerts for Disaster/Crime to the twitter user as well as local agency for them to investigate/help. It converts the unstructered web data into structured format by extracting, preparing and integrating web data in areas of crime, natural disasters, etc for consumption in criminal and GEO helping investigation agencies. It takes full ability of Twitter API to fulfill its job.
TWEELP also provides a visual environment for automating the workflow of extracting and transforming web data using the twitter api. After gaining insights from the tweets of people from a particular location where disaster, crimes etc started to gain interest, the web data extraction module provides a visual environment for designing automated workflows for harvesting data, going beyond HTML/XML parsing of static content to automate end user interactions yielding data that would otherwise not be immediately visible.
Once extracted, the software provides full data preparation capabilities that are used for harmonizing and cleansing the web data.
For consuming the results, TWEELP provides several options. It has its own visualization and dashboarding module( which includes filtering & AI processing of media) to help criminal investigators gain the insights that they need. It also provides REST APIs that offer full access to everything that can be done on our platform, allowing web data to be integrated directly.
TWEELP uses NLP to cleanse the data. The AI Model is capable of geolocating the probable crimes, natural disasters etc.
Scheduler is a sub application developed for TWEELP to automatically recheck the authenticity of existing disasters or crime so that if any new report is found, it checks whether the old report was correct or not. By default scheduler runs after every 3 days which can otherwise be set to the user's choice.
How I built it
I started building the basic need of the project which requires setting up an alert mechanism based on the result of our AI Model which categories the tweets into Disaster/Crimes etc.
To make it more robust, I created a report file which can be maintained for future use and can even be scheduled to recheck the authenticity of the event that has happened.
The TECHSTACK for TWEELP is: Django, JS, HTML/CSS, NLTK, tweepy, Twitter API
Challenges I ran into
Coming to an idea, that things are not stable in life, even a stone would change it location if wind blow too hard. So for developing something that would tell us the status of same query every time after X number of days, was a good challenge for us. I came up with the scheduler, which would automate the process after every X days and tell us the authenticity of the old reports, which could conclude whether the tweets are just an rumor or real.
Accomplishments that I'm proud of
Developing complete end to end platform which could not just help Investigating Agencies, but also save lives by alerting individuals. I have added security patches that could protect us from commons attacks. Making it Multilingual so that different communities can be able to use it.
What I learned
Time Management, thinking better in tough scenarios.
What's next for TWEELP
Getting our AI model more accurate, more flexible with more added sub categories for Natural Disasters, Crimes, Woman Abuse, Child Abuse etc. And making TWEELP more diverse for different users.