EduWay

After trip score
Progress - Last trip events
Past Drives
Home screen
Progress - Score trend
Replay - Infractions
Replay - Route
Driving Growth - Gamified Experience
AI Coach - Chris

Inspiration

Drawing from previous experiences in post-game Gainesville where team members have almost been run over by reckless driving, we recognized this as a problem that we wanted to tackle. When considering what tracks to go for, we realized that Waymo's and State Farm's tracks feature transportation safety as a subject, something that also influenced our decision.

What it does

EduWay encourages new student drivers to develop better driving habits by detecting suboptimal behaviors, gamifying progress towards better driving, and providing personalized feedback on trips. Using sensors onboard the phone and data from OpenStreetMap, we are able to detect various reckless driving behaviors, such as speeding, hard braking, sharp turns, among others. These detected errors are verbalized live to the user using ElevenLabs and logged in MongoDB for future retrievals, and after a trip is done, the app assembles a summary of mistakes made and provides a score. Then, this data is fed into Gemini, which alongside ElevenLabs and Google Street View, provides the user with a personalized and narrated improvement plan. This feature stands out because the coach also remembers past trips by pulling aggregated data from MongoDB, so it is able to identify recurring errors and locations and verbalize these to the user. Users are also provided with a map of their route in which they can play their trip back and see what mistakes happened where. Outside of rides, users can track their score progression and earn streaks for maintaining high scores through various trips.

How we built it

We built it using spec driven development, with individual contributors having clearly defined tasks up until the minimum viable product. Individual contributors used coding agents and rigorous testing to ensure timely feature delivery.

Our stack uses Expo libraries to gather phone sensor data, GPS location, audio playback, and other phone functions. Our thresholds work regardless of how the phone is mounted because of our filtering and detection pipeline, that was validated with both real and synthetic data. We use OpenStreetMap via the Overpass API to look up speed limits and stop signs. We use React Native libraries to draw our maps and show the Street View panorama page.

Our backend uses Express, which serves the REST API for trips, traces and results, while MongoDB stores trips, events, GPS traces and user progress. Gemini handles the coaching debrief after each trip and powers the coaching chat, while ElevenLabs turns that brief into speech and generates the fixed live alert clips ahead of time.

Challenges we ran into

We originally wanted to use Google's Roads API to pull speed limit info, but this required an Asset Tracking license that we would have been unable to obtain in the timeframe of the hackathon, so we switched to OpenStreetMap.

By virtue of the project being a mobile app that used onboard sensors, having it be accessible as a web app, even for demo purposes, would be a severely degraded experience. We aimed to have a deployed Express instance serving a single QR code that links to Expo Go for the purposes of demoing, but in the process of getting this ready we ran into some issues where the app would not load at all, even if the user connecting was verified.

Accomplishments that we're proud of

Validating our thresholds for detecting errors involved gathering live data in a practical scenario, so we, within the limits of reason and legality, collected data from the field. We also had to extrapolate some thresholds. For example, our signal for swerving is a multiple of lane switching signals. To normalize our scoring, we pulled field gathered and synthetic routes, gave them ideal scores, and used fitted models to derive the weights of each mistake and how they vary with the length of the trip.

Our initial idea and scope were determined by extensive discussion, and rather than committing to the first idea that came to mind, we kept refining or coming up with new angles to our problem.

After finishing our MVP and heading to a team member's house to rest, we tested our app on the way there. The trip took almost 50 minutes, and our app was able to keep track of 72 different mostly minor events (speeding has cooldown), 10 of which were severe, provide a reasonable score, and run every part of our "after action report" the way we intended it to.

What we learned

We learned about preventing scope creep, combining data from different APIs, using MongoDB to power RAG, creating structured prompts for Gemini API, collecting sensor data from phones, and increasing the visibility of core features.

What's next for EduWay

Were we to expand upon this project in the future, we would try to have more gamification features, such as badges and weekly challenges. We would also implement a notification system so that summarized "coach" impressions are served to the user when they are ready, and implement a dashboard for parents to look at the progress that a student chooses to share. Finally, we could add support for detecting more types of reckless behavior.