Inspiration

USC is a campus with renowned faculty and highly motivated, and hard-working students from around the world; but USC is also known to be centrally located around a high crime rate neighborhood. We aim to help our university community and public servants such as the Department of Public Safety and law enforcement by creating a median that can detect high threat risks. We are determined to make sure that everyone feels comfortable to go to school, to go to work and safe. Thus, born -- The Guardian.

What it does

The Guardian will be capable of detecting high threat risks like gun shooters before they cause any type of damage by identifying objects that are shaped like guns. The Guardian will also be capable of detecting bullying on various spectrums ranging from verbal to physical (physical coming soon). Especially, the Guardian is our beloved Baby Yoda.

Gun detection: Baby Yoda can detect various guns and will alert everyone via a mass text message and email immediately after detecting any gun holders present on campus. It will send a signal to the server allowing local law enforcement to manually control Baby Yoda and apprehend the gun-wielding suspect. This allows some type of control in the scene rapidly through Baby Yoda while backup arrives to further alleviate the situation. Baby Yoda will also be capturing photos to send the suspect's image to our website and keep everyone who uses the application updated and aware of the current situation and know what areas to stay away from due to the high risks.

Bully detection: The Guardian will be able to speak in different tones based on the level of bullying occurring. It will also draw the attention away from the bullied victim and draw it to itself to allow the bullied victim to either getaway or stop getting attacked. Baby Yoda will be able to detect many types of bullying be it from profanity, sexual harassment, personal or verbal attacks, and even criminally type comments. Baby Yoda will be able to locate the suspect who is being the bully and warn them to politely respect the personal space of those individuals being threatened by the bully. The person getting bullied will be protected by Baby Yoda and reassured that everything will be okay and that additional help is on the way if it is required.

How we built it

Gun Detector: We implemented alwaysAI to deploy our very own trained TensorFlow Convolutional Neural Network (CNN) model to be capable of detecting guns such as rifle/AK-47 based on their structure, shape, and size. We used the edge API of alwaysAI to run that model onto the Raspberry Pi. We included GCP's facial detection to capture a snapshot and identify the gun suspect from the image.

Bully Detector: We used GCP's speech to text to analyze common words spoken by people on a daily basis then used Tisane API to filter out and detect words, phrases, and sentences considered to be insulting and or inappropriate. We also use text-to-speech to create a speaking feature for Baby Yoda so it can speak up for the bullied victims as well as cheering them after the incident.

Web Application: We used ReactJs to design the front end and to display the image of the gun suspect, location of occurrence, and time of the incident. We created the backend server using NodeJS and used MongoDB for the database.

Challenges we ran into

Software:

  • AlwaysAI is hard to deploy due to strict configuration and the way to train the model is different from other AI models we have experienced before. We managed to overcome some of the hurdles but with more practice see this as an issue that can be fixed.
  • We did not have a microphone for our raspberry pi which led us to have to hack it by using one of our members' laptop to use it as a "microphone" via the laptop, and we used MQTT Broker protocol to send real-time data to the Raspberry Pi.
  • We spent a large amount of time designing the hardware model. In doing so we had a hard time calculating the center of mass of Baby Yoda's base. The wheel receives a lot of gravity force and weight from the power bank which reduces the speed of the motors we are further looking for solutions to this problem.
  • The communication between the Raspberry Pi and Arduino via serial communication sometimes was delayed.
  • We really found issues using the Google Authentication issue. The export GOOGLE_APPLICATIONS_CREDENTIALS kept bugging us for about 8 hours. After this, we are determined to work with Google to fix this problem and avoid problems like this from ever occurring again.
  • After the round with Google, we faced another authentication issue but now from MongoDB.

Hardware:

  • Using a lot of hardware and connecting them via multiple servers gave us some issues. One of the biggest issues was that the latency of connection was large and we are diligent in fixing this issue to make it more responsive.
  • We also have a lot of components inside Baby Yoda which makes it very heavy and slows it down.

Accomplishments that we are proud of:

  • We are proud that we stuck together overcoming complications till the end as a team. This bonded our friendship really close as we trusted one another to complete a task given (Spoiler Alert: one person didn't even sleep).
  • We really fell in love with this project a lot since we combined our knowledge of hardware and software together.
  • Also, we built Baby Yoda from scratch and struggled together to raise and bring up the baby. It is quite breathtaking seeing something come up from nothing to the vision you and a group had.
  • One of our members discovered an interest in sound engineering and creates many unique and impressive Japanese character voices from bytes array buffer.
  • Making many different components in this project merge together and come to life and actually work is something our team is proud of. There were moments where bugs were intimidating us but we overcame them together and brought our baby to life.
  • We did our first ever 3D print which is somewhat successful for our first time ever playing around with the software and learned something new in the process.

What we learned

  • Our team members, John Marshall and Ellie Huang, created their first ever NodeJS server and learned a lot of valuable debugging skills in the process. They also struggled with MongoDB since it is NoSQL and different from the traditional MySQL and learned a lot of valuable skills in the process. They also learned that Django isn't quite useful in situations like we were found in.
  • One team member learned how to use a 3D printer, the software to create pieces, and how to use a laser cutter.

What's next for Guardian of USC

  • Automate the Guardian to chase the gun shooter itself.
  • Make Baby Yoda use the Force Awaken to attack the shooter himself with a stun gun, which can stun the shooter to prevent damage to anyone and or anything around their surroundings.
  • Make Baby Yoda's voice more childlike ( like a young Justin Bieber =P)
  • Improve the UI/UX of the React
  • Sleep more and get ready for the next hackathon!

Miscellaneous

  • Other member had a headache but he still "FIGHT'S ON" till the end.
  • We also watched Reinforcement Learning AI for fun since we were stuck with bugs for quite some time

Built With

Share this project:

Updates