BikeGuard

Logo
yolov8 training
bolt-cutter tuned model

Inspiration

Seeing many posts and stories of people losing their bikes and many times their only reasonable way to travel in time, we wanted to create something that could protect these people from both emotional and financial loss.

What it does

The mission of BikeGuard is to identify stolen bikes in real time, alerting owners as fast as possible. BikeGuard uses Gemini powered image and speech recognition to accurately detect such threats through three major components:

The user may upload multiple past logs in the upload section, parsed through Gemini to detect bike theft. This is useful in cases where the bike was left unattended for large periods of time. The saved logs end up in library where a running count of umber of threats is present.
The user may also take advantage the multimedia model, through its language detection capabilities, able to infer if a bike is stolen in a noisy environment. This is paired with the alert feature, where an automatic email alert is sent to the owner
Finally, three triple live camera feature allows the live analysis of footage in three remote locations. The green and red boxes make it easy to indicate wether the bike was stolen and if theft is detected, the model will again alert the user through email.

How we built it

We built BikeGuard by using React and Vite on the frontend. for the backend, we used our own fine-tuned yolov8 model, coco yolov8, and Gemini for the AI detection, and firebase for the sign-in and email sending capability.

Challenges we ran into

Both parsing through the Gemini API and training the yolo v8 model were tedious. Gemini API only accepts limited and small video formats-specifically h.264, so we had to compress our files in the unique format. Additionally, for both models, especially yolo v8, finding clean data without shading with only relevant objects appearing, was challenging. For training yolo, lots of labeling was done for 50+ pictures. Additionally, having Gemini send out an alert and integrate with the speech to text was hard but rewarding.

Accomplishments that we're proud of

We're proud of getting the API calls working with Gemini and firebase, and the model we trained. We are also proud of how the website looks.

What we learned

We gained lots of experience with react, ML training and integration, and firebase. For example, the trailing text on the home page required us to learn new commands. Additionally, the trailing text from speech to text was fun to implement and learn. The hologram bike animation taught us how to import moving images in react without background, creating a unique user aesthetic. We learned how to train ML models, from data collection to labeling, and through train/val/test splits. Learning firebase taught us about sign in tools and email collection, allowing us to integrate a google login and auto email.

What's next for BikeGuard

We will train the yolov8 model in order to have better/more detection capabilities.
We also want to include heuristic rules with people memory to have a more nuanced threat alert, i.e., if person B is close to person A's bike for too long, raise suspicion. In the future, we also want to expand this to scooters and other personal vehichles