Roadsense AI

Inspiration

We have all felt that sudden jolt when a vehicle hits a pothole—and we have all seen how poor road conditions lead to accidents, delays, and vehicle damage. During Smart India Hackathon, we kept asking a simple question: what if every smartphone or dashcam on the road could become a real-time road inspector? That idea became Roadsense AI. We were inspired by the possibility of using technology people already carry—cameras, GPS, and motion sensors—to solve a massive public problem without requiring expensive new infrastructure. Instead of waiting for complaints or manual surveys, we wanted to give authorities and drivers a live, data-driven picture of road health and risk.

What it does

Roadsense AI turns ordinary vehicles into intelligent road-monitoring nodes. Using a dashcam or smartphone camera, along with GPS and accelerometer data, it detects potholes, cracks, open manholes, speed breakers, waterlogging, and accident-prone stretches. It then combines visual defect confidence, vibration severity, accident history, and traffic or weather impact into a weighted Road Risk Index. The system alerts drivers in real time, generates heatmaps for municipal authorities, and prioritizes repairs based on severity. It also detects harsh braking and near-miss events, helping identify dangerous zones before accidents happen.

How we built it

We started by collecting dashcam videos, accelerometer logs, and GPS traces from real roads. We labeled potholes and road defects using Roboflow and CVAT. For vision, we trained YOLOv8 for object detection and a U-Net variant for segmentation. For sensor time-series, we used a 1D-CNN plus LSTM model to classify vibration patterns. On the edge, we converted models to TensorFlow Lite for Android. The backend used FastAPI, PostgreSQL with PostGIS, Redis, and MQTT for IoT streams. The dashboard was built in React, while the mobile app used Flutter. We smoothed GPS drift with a Kalman filter. Everything was containerized with Docker and deployed on AWS.

Challenges we ran into

We faced data scarcity and severe class imbalance—potholes are rare compared to normal road frames. Shadows, rain, night lighting, and speed breakers confused the vision model. Running inference in real time on low-end Android devices forced us to quantize and prune our models. GPS drift made location accuracy unreliable, so we relied on sensor fusion. False positives were a constant battle. We also had to anonymize footage for privacy and design APIs that could eventually integrate with government systems. Coordinating across team members and tight SIH deadlines added another layer of complexity.

Accomplishments that we're proud of

We built a working end-to-end prototype that detects road defects, computes risk scores, and displays live heatmaps. Our edge model runs on Android with low latency, and our sensor-fusion pipeline reduced false positives significantly. We created a dashboard that lets authorities filter by severity, location, and repair priority. Most importantly, we proved that a scalable, low-cost road intelligence system is possible using tools people already carry.

What we learned

We learned the full ML lifecycle: data collection, labeling, training, optimization, deployment, and monitoring. We learned that edge AI is as much about constraints as accuracy. Sensor fusion taught us to think beyond a single data source. We also learned that civic technology needs trust, privacy, and usability—not just good models. Finally, we learned how to pitch a complex idea clearly and work as a team under pressure.

What's next for Roadsense AI

Next, we want to pilot Roadsense AI with municipal corporations and smart city command centers. We plan to add V2X communication, crowd-sourced reporting, and predictive maintenance. We also want to expand into accident prediction, insurance partnerships, and integration with navigation apps. Our long-term goal is simple: make every road safer, smarter, and more accountable for every Indian.