Inspiration
Every monsoon, India watches the same tragedy unfold. Newly paved roads disintegrate, causing fatal accidents and crippling city traffic. In 2022, a massive accident on the Mumbai-Pune Expressway claimed multiple lives when an SUV plunged into a water-filled crater-a pothole that had been "repaired" just three weeks prior. We realized the root cause isn’t just bad weather; it’s "blind payments." Today, drones inspect these roads, but they only bring back 2D photos. A flat image shows where a road broke, but hides how deep the structural failure goes. Contractors exploit this blind spot, billing for 10 tons of asphalt but only pouring 2. We are paying full price for dangerous illusions. We built InfraTwin AI to stop this cycle of corruption and neglect.
What it does
InfraTwin AI is a 100% software-based SaaS platform that turns standard commercial drones into forensic auditors for road infrastructure. 3D Volumetric Auditing: Instead of 2D bounding boxes, our AI fuses YOLOv8-Segmentation with Hugging Face’s "Depth Anything" model to calculate the exact 3D volumetric displacement of missing asphalt. Instant Fair Budgeting: A GenAI Copilot (Llama-3 via Groq) takes the volume data, cross-references Indian Road Congress (IRC) standards, and instantly generates a kilogram-accurate repair budget (Bill of Quantities). Anti-Corruption SLA Verification: When contractors claim a road is fixed, our AI compares the "Before" and "After" drone scans. If the billed material doesn't match the actual volume filled, the system flags the fraud and freezes the payment.
How we built it
Frontend & Spatial UI: Next.js, Tailwind CSS, Shadcn UI, and Mapbox GL JS for the interactive digital twin dashboard. Backend: Python, FastAPI, and OpenCV for asynchronous video frame extraction and routing. Vision AI Pipeline: Ultralytics YOLOv8-Seg for pixel-perfect semantic segmentation (Area), and Hugging Face "Depth Anything" for monocular depth estimation. We built a custom math engine to multiply Area × Depth to yield the 3D Volume. Cognitive AI (GenAI): LangChain and Llama-3 (via Groq for real-time, millisecond responses). We used Pinecone Vector DB to store and retrieve Indian Road Congress (IRC) standards using RAG. Databases: PostgreSQL for relational user/project data.
Challenges we ran into
The biggest challenge was scaling 2D pixel data to real-world 3D volume without expensive LiDAR hardware. Monocular depth models output relative depth maps, not absolute meters. We had to engineer a mathematical scaling logic that used the known dimensions of standard road elements (like lane markings) as a reference point to accurately convert pixel depth into real-world cubic meters.
Accomplishments that we're proud of
What we learned
We learned that the real value of AI in infrastructure isn't just in finding the damage—it's in auditing the money. Technically, we learned how to chain multiple heavy AI models (YOLO + Depth + LLM) without destroying the user experience by using asynchronous API calls and fast inference engines like Groq.
What's next for Infra-Twin AI
- Predictive Maintenance: Training time-series ML models on historical degradation data to predict which "good" roads will fail next before the monsoon hits.
- ESG Carbon Tracking: Integrating a carbon-footprint calculator into the BOQ to help governments track the CO2 emissions saved by doing precise, preventative repairs rather than full excavations.
- Drone SDK Integration: Partnering with DJI to allow direct automated video uploads from the drone to the InfraTwin portal, removing the manual step entirely.
Log in or sign up for Devpost to join the conversation.