driveX

• 

  Home Page

• 

  Info Page

• 

  Driving Page

• 

  Alert Page

Inspiration

Three days ago, one of our teammates and her friend went on a beautiful hike to Lake Tahoe. Driving back to San Francisco late at night, they felt sleepy and tired and had to talk to each other to stay awake. Luckily, they both were able to get home safely, unlike approximately 230,000 people each year who get into car accidents at night due to drowsiness. Approximately 3% of these car accidents are fatal.

Driving at night puts drivers and passengers at risk, especially when the driver is alone and has no one to talk to. This problem inspired us to explore Computer Vision solutions to help drivers have a safer driving experience at night. We decided to implement an innovative solution that tracks a driver’s face to detect drowsiness and alert them, as well as communicate with the driver via natural voice to keep them engaged.

What it does

driveX is a tool that helps people experience safe night driving. Its safety features include a real-time alert system, which triggers a sound and notification when it sees that the driver is falling asleep or looks tired. It uses a live camera and when it notices that the person is drowsy, it will immediately show this alert and play the sound continuously until the driver acknowledges the alert. It also includes a personal driving assistant, which helps keep the driver engaged during the ride. It can chat in a natural voice with the driver about their day or talk about topics of their interest to help them stay awake.

How we built it

After initial ideation, we moved to the design stage during which we utilized Figma software to create a user-friendly UI. For the UI design, we focused on using colors that are comfortable for users to see at night and designing components that allow drivers to safely utilize the application while driving.

For our main logic, we conducted research on suitable Computer Vision and Large Language Models to implement eye tracking and conversational components of our application. We found two models on Hume AI that perfectly fit our use case: the Expression Measurement API for tracking facial expressions to detect drowsiness and the Empathic Voice Interface for natural conversation, which encourages drivers to think and talk, thereby reducing sleepiness. For the Expression Measurement API, we chose to utilize several features to detect sleepiness, including eye closure, head movement (head down), mouth movement (yawning), and tiredness level. These features provide a reliable prediction of when the driver is about to fall asleep, helping to prevent accidents.

As a final development step, we created the main pages using TypeScript and Next.js and integrated the chosen APIs. We conducted extensive testing to ensure the application's reliability and user experience, addressing any issues that arose. Our application is now well-equipped to enhance driver safety by monitoring drowsiness and facilitating engaging conversations to keep drivers alert.

Challenges we ran into

One challenge we ran into was defining accurate thresholds for different features that detect tiredness or drowsiness, which trigger the alert. We spent a lot of time experimenting with different values and combinations of features (like "eyes closed", "tiredness", "head down", and "mouth open" [for yawning]) that the computer vision model returned to see what conditions would best define the conditions to trigger an alert.

Accomplishments that we're proud of

Our team is proud of creating an application that can significantly enhance road safety at night. We believe that our application will help drivers stay alert during their journeys, reducing the likelihood of car accidents. From a technology perspective, we are proud of our successful integration of cutting-edge Computer Vision and Large Language Models, as well as our focus on creating a user-centered design.

What we learned

Our team gained valuable experience in using computer vision models, specifically for real-time facial expression analysis to detect driver drowsiness. We also enhanced our skills in integrating AI-driven APIs and developing user-friendly interfaces. Building on our previous knowledge of front-end development, we expanded our expertise in working with video and camera technology, ensuring seamless integration and real-time processing capabilities. This project allowed us to deepen our understanding of combining front-end design with more advanced techniques, including computer vision and large language modeling, ultimately contributing to an AI powered user-centric application.

What's next for driveX

In the future, we see driveX as not just being a web or mobile app, but also a tool that can integrate seamlessly with autopilot services in autonomous vehicles. For example, in addition to playing an alert that the driver must acknowledge when the camera detects tiredness or drowsiness, the autopilot system can take control while the alert is triggering. The system can also integrate well with the AI driving assistant, which chats with drivers to pass time, ensuring they remain aware of their environment. This integration of driveX into autonomous systems of modern vehicles can provide a safer and more connected driving experience with its real-time alert system and AI personal driving assistant.

Built With

• figma
• humeai
• next.js
• react
• tailwindcss
• typescript