BotPatrol

Bot Patrol Right Side
Bot Patrol Back Side
Bot Patrol Left Side
Bot Patrol Front

Inspiration

ILLINI ALERT, sounds scary, right? We have around 45k students here at Illinois and due to the rigorous course load, students are often up late studying at libraries and educational buildings. Even though our Police are highly active and responsive, it is only viable for them to patrol some possible places on our huge campus. Security is a massive issue not just in Illinois but across the nation. Even though there have been technological advancements towards it (CCTVs), we need something better. With something to patrol your building/residence 24 hours, 7 days a week, you can sleep peacefully and study stress-free. Next time you are at Seibel grinding out your MPs, you should not be scared of an intruder entering the building but just be stressed about failing the MP.

What it does

As its names suggest, Bot-Patrol is an autonomous vehicle designed to patrol a designated area. Whether residential, commercial, or educational, Bot-Patrol provides top-tier security. The user can draw their personalized path around the area that they want to be protected and pass it to our Bot-Patrol using the beautiful web interface. The Bot-Patrol follows that defined path unless it identifies something fishy. More specifically, the Bot-Patrol will look for hindrances in its path; upon finding a hindrance, it will turn towards it and analyze it. Should it detect a human presence, it tactically approaches to deter the individual, thereby ensuring the safeguarding of the designated area. Additionally, we provide an even more personalized Bot-Patrol where the user can change the sensitivity of our hindrance detection system. For instance, if the user lives in a busy neighborhood where people are walking near your residence at any given time, you would not want your bot to move towards each person passing by; instead, you can decrease the sensitivity to only detect people who are really, I mean realllllllly close.

How we built it

We started with the starter code that John Deere provided us. Using this, we started getting our MVP working directly on the Raspberry Pi through the ssh directions John Deere gave us. So, we ensured our bot could go in and out of a set holding pattern, detect objects to the left and right, and detect people straight ahead. Once we had this all written in Python directly on the Raspberry Pi, we started moving into newer features. We used HTML and CSS, along with a flask backend to create a user interface. This user interface would allow the bot to start from anywhere with multiple holding patterns. For the software part, we started by researching different image recognition algorithms. Upon testing a few, we locked in on the YOLOv8 image recognition machine learning algorithm since it gave the best real-time analysis with the highest accuracy and confidence levels. We utilized the algorithm to obtain a binary output classifying whether the image contained a human or not. Once the image recognition was up and running, we were focused on getting the bot to move toward where we got the ultrasound signal from. For this, we prioritize between the two sensors we have based on distance and rotate the bot toward our prioritized direction. Then we pass an image frame to the bot captured using the camera and our image classifier detects if the object detected by the ultrasound sensor is a human or not. If it is a human, we move towards the human for one second and then return to our original path. If it is not a human then the bot returns and continues along its original path.

Challenges we ran into

We wanted our Bot-Patrol to run on a predefined path but also to leave that path if any unusual activity was detected, we were running into issues about maintaining the path since we also did not have GPS. To solve this problem, we defined our path in terms of time and maintained a global time variable which was paused everytime we moved out of our path.

Additionally, the motors we were provided were not equally powered, affecting our code to move forward and backward. We had to normalize the speeds of both the motors and fine tune it to move in a straight path.

Accomplishments that we're proud of

We’re proud of getting our AI model working directly on the raspberry pi. Using this, we can tell if the object detected was a human. This lets us make many more unique decisions based on what we can see, allowing us to react more seamlessly.

Further, we are really proud of all of the logic we implemented, letting the bot work independently without any consistent input. We expected this to be very difficult to determine how to handle all the inputs, and it was. So the fact that we could get it to work made us very proud.

What we learned

One thing we learned was how to work with hardware directly, especially with all of the limitations of the hardware. For example, we were given two motors, which did not run at the same speed for different powers. So, we had to do a lot of testing to ensure we could consistently get our bot to move without erroneously turning.

Another thing we learned had to do with optimizing our code. We had to make tradeoffs with how often we processed images in order to have our code run faster, and make things more seamless.

What's next for Bot-Patrol

Implement GPS Tracking Currently, we only have a predefined path as a straight line, square, or triangle; however, if we manage to incorporate GPS tracking into it, we can let the user better customize their path to fit the area they are trying to secure.

Object Detection We have only implemented a binary image classification algorithm that identifies the obstacle as human or not. However, we can increase the scope by classifying dangerous and non dangerous objects. Furthermore, if we fine-tune the model enough, we can identify humans with dangerous objects such as a pistol or a knife. Taking Action Currently, if we find a human within our defined proximity, the bot will move one step towards them and return to its original path, supposedly scaring the person off. We can take several actions in real life, such as alerting the bot's owner, notifying the nearest police station, or turning on an alarm system. This will ensure enhanced safety and security.

Facial recognition for finding suspects On a further scale, we can use our Bot-Patrol to aid the police force in searching for criminals on a spree. We can place the bots near the crime scene, the suspect's house, or any other place the police suspect the criminal to be and pass the image of the suspect to the bot. The bot will look at every human crossing by that area and configure if the person matches our suspect.