Shelter Helper

Developing location tracking and deep learning based solutions to help the homeless

Inspiration

Homelessness is a major problem in the nation. A report published by the White House addressed this with 552,830 people being counted as homeless in the year 2019 in the entire country. There are thousands of NGOs & Shelter homes working for the upliftment of these people but the results are not satisfactory. False reports of homeless, inefficient tracking of the rough sleepers and no co-operation between various NGOs drove us to use technology oriented solutions to function for the greater social good.

What it does

The Shelter Helper Mobile site greets you with a home page consisting of prompts for either reporting a homeless or asking for help. If you report a homeless person, you can just click a picture and give location access. Shelter Helper takes the image and sends it along with the label which tells the probability of the picture being of an actual person in distress. The coordinates of the place where the picture was clicked are sent to a nearby NGO who can then take further required steps.

How we built it

We used HTML, CSS, Javascript and BootStrap to develop the UI of the site which was backed by a Django framework. Geolocation libraries (GeoDjango, Postgres-Postgis and HTML-Geolocation API) were used in Django to extract the location of the consumer of the app, and sent the extracted image to our deep learning model. We ran our convolutional neural network which was trained using transfer learning over a dataset of just 120 pictures which we data augmented to get a dataset of 1920 different images and performed parameter tuning iterations to train it to 95% + Accuracy.

Challenges we ran into

We faced several problems throughout the process of starting everything from scratch till integrating the whole project. Firstly, we faced some issues in our UI, majorly due to lack of proper hardware as our web designer’s laptop was left at the university. We made do with inefficient monitors and faced a lot of issues in putting the JavaScript based graphs on our UI. One major problem we faced was in the backend part, extracting the location of the user. It would have been an easy task if we had access to Google Maps or Apple Maps API but as we had no such resources, we used a geolocation library (Postgis) which was difficult to integrate to our database to take the coordinates. We could not host our website due to the model size and lack of server space. Moreover, it was a challenging task to create a server to connect our mobile site to our neural network.We built our server on django, and faced some issues when trying to load the model for prediction. Since neural networks take a long time to train and predict, we knew we had to create a server to query our pre-trained model for class labels. The model itself was trained on a small dataset which we scrapped manually. This was however covered up with heavy use of data augmentation (color jitter, random crop, etc.)

Accomplishments that we’re proud of

Ultimately, the best thing over the course of the Pyghack 2020, was us collaborating over a period of 48 hours, trusting each other, improvising on our mistakes and facing challenges together. Completely compiling a fully functioning mobile site with sophisticated backend and model integration in such a short span of time is something we’ll cherish whenever we’ll think of the time when we participated in a hackathon, remotely, in the middle of the pandemic. What we learned : In the social aspect, sincere analysis of the problem in frame and realising the seriousness of the issue helped us understand one of the many social problems people face in day to day life which everyone cannot easily relate to. On the technical counterpart, we had an idea how to develop our parts but bringing the whole site to life and integrating all the minor aspects to develop a beautiful (both aesthetically and socially) end product helped us learn a lot.

What’s next for us

We can expand by first connecting with neighbouring shelter homes. By having an exhaustive database of such NGOs we will be able to give better results. As the backend involves running a deep learning model, it takes a lot of time for computation locally. We can use services like AWS or Azure to upload our model as well as the database online for easy and expandable usage. This will save on computational time, making the backend more lightweight. If the idea gains traction, we can collaborate with surrounding states and counties to contribute towards the social cause.

The Team -

Shashvat , Sajal , Karan and Aryansh

Contact us at

karan.uppal3@gmail.com

Share this project:

Updates