Inspiration
Autonomous vehicles are being developed around the world, but India presents a very different driving environment. Indian roads can contain dense two-wheeler traffic, auto-rickshaws, pedestrians, livestock, unclear lane boundaries, temporary obstacles, informal signaling, and rapidly changing road behavior.
We asked a simple question:
Before an autonomous vehicle enters India, can we test whether it actually understands India?
That led us to build DesiDrive AI, an India-specific testing and decision-support layer for autonomous vehicles and ADAS systems.
What we built
DesiDrive AI analyzes road scenes using computer vision, trajectory tracking, multi-agent interaction analysis, audio context, and Google Gemini multimodal reasoning.
Instead of only asking "What objects are present?", DesiDrive asks:
"What are these agents likely doing, how are they interacting, and how difficult is this scene for an autonomous driving system?"
The system produces:
- India Road IQ
- India Autonomous Readiness
- Multi-agent interaction graphs
- Explainable Safety Analysis
- Pedestrian and livestock risk analysis
- Two-wheeler complexity analysis
- Horn/audio context
- Evidence linked to calculated interaction metrics
- Human feedback for incorrect predictions
- A 3D simulation lab with multiple camera views
Gemini receives actual visual frames together with structured perception and interaction context so that scene reasoning is grounded in both visual and calculated evidence.
How we built it
The frontend is built with React and Vite. The backend uses Python and FastAPI. OpenCV and object-detection/tracking components provide perception and trajectories. A deterministic interaction engine calculates trajectory relationships such as time-to-approach and closest-point-of-approach. Gemini provides multimodal scene reasoning, while a local fallback keeps the prototype usable when the API is unavailable.
We also created controlled synthetic scenarios for:
- Two-wheeler filtering
- Pedestrian + mixed traffic
- Livestock / unexpected obstacle
These synthetic scenarios are clearly separated from real-world data and are used to demonstrate and validate the system.
Challenges
One of the biggest challenges was making the system reliable outside ideal driving conditions. We had to handle uncertain detections, duplicate objects, multimodal inputs, offline fallback behavior, trajectory interactions, and deployment differences between a local Python backend and a web frontend.
We also learned that an impressive AI demo is not enough — every displayed risk or metric needs to be traceable to actual evidence.
What we learned
The biggest lesson was that autonomous driving in India is not simply an object-detection problem. Behavior, interaction, uncertainty, and context matter just as much as recognizing the objects themselves.
DesiDrive AI is a prototype testing and evaluation platform, not a vehicle-control system. It does not issue steering, braking, or acceleration commands.
What's next
Our long-term vision is to make DesiDrive a standardized India-readiness testing layer that can evaluate autonomous-driving systems across diverse Indian road scenarios before real-world deployment.
Built With
- antigravity
- gemini
Log in or sign up for Devpost to join the conversation.