RoadBuddy AI

Inspiration

We all experienced feeling tired and sleepy when driving for long distances especially driving back home after work or school. We understand the huge risk behind driving tired or sleepy, however, many people take it. That's why we decided to build RoadBuddy AI.

What it does

It monitors the driver's face and head movements in real time to detect early signs of being sleepy and tired. If these signs are detected, it then engages with the driver based on their history and interests. Drivers can pick one of many voices available with different tones and accents. Engagement depends on how dangerous the situation is. It ranges from checking on the driver and starting a small conversation, and directing them to the nearest coffee shop, to sending alerts asking the driver to pull to the side when safe and stop driving.

How we built it

RoadBuddy AI is built using Vite + React for the interface, Python for the backend, MediaPipe for real-time face/head detection, Google Gemini for the chat with the driver, and ElevenLabs to interact with the driver in a more natural way, and MongoDB for the database to store all the logs, calibrations, user details and history. A personalized calibration tool is used to ensure that no matter where the camera is placed and who is using it, it remains always accurate. This tool is light weight and privacy focused as it runs offline and does not store any pictures or videos.

Challenges we ran into

Getting reliable and accurate information using MediaPipe. Making engagement with the user feel more natural and less like an automated robot voice.

Accomplishments that we're proud of

Personalized calibration for each user and location, this makes our tool way more accurate, with preset accurate thresholds that work for every user. Smart and context aware engagement and not just alarms

What we learned

Calibration is very important when using real time computer vision.

What's next for RoadBuddy AI

Optional cloud sync for families or companies for their fleets. Integrate directly with car systems and be able to get data such as speed and braking.