BuildNova

Inspiration

The construction industry continues to suffer from costly rework, delayed inspections, and safety incidents due to manual site monitoring and fragmented project information. Existing solutions primarily visualize BIM models but lack the ability to understand, reason, and proactively assist workers. We envisioned BuildNova as an AI-powered construction copilot that brings intelligent decision-making directly to the construction site.

What it does

BuildNova is an AI-powered construction assistant that combines Vision-Language Models (VLMs), Building Information Modeling (BIM), Retrieval-Augmented Generation (RAG), and Augmented Reality (AR) to help workers and engineers make better decisions in real time.

It can:

Scan and understand the construction site. Compare live progress with BIM models and project drawings. Detect construction deviations and potential safety hazards. Retrieve project-specific documentation and standards using RAG. Provide contextual guidance through AR overlays. Assist workers using natural language queries. Automatically update project progress for supervisors.

How we built it

We designed BuildNova as a modular AI system consisting of:

Vision-Language Model for scene understanding and visual reasoning. YOLO + Computer Vision for object detection. BIM Integration using IFC models for comparing construction progress. RAG Pipeline to retrieve engineering drawings, manuals, and safety documents. Multi-Agent AI for quality inspection, safety analysis, and planning. AR Interface to display real-time guidance. FastAPI backend, React frontend, and PostgreSQL/Qdrant for data management.

Challenges we ran into Aligning real-world construction elements with BIM models accurately. Handling complex construction scenes with multiple overlapping objects. Designing AI outputs that are explainable and trustworthy for engineers. Integrating multiple AI components into a seamless workflow. Ensuring the system remains practical for real-world construction environments.

Accomplishments that we're proud of Developed an end-to-end concept that combines VLMs, BIM, RAG, and AR into a single construction intelligence platform. Designed an explainable AI workflow that doesn't just detect problems but also explains why they occur and recommends corrective actions. Proposed a multi-agent architecture capable of assisting workers, inspectors, and project managers simultaneously. Created a scalable solution that can evolve into a full digital construction assista

What we learned

Through BuildNova, we gained a deeper understanding of multimodal AI, computer vision, retrieval-augmented generation, BIM workflows, and AR applications. More importantly, we learned that combining multiple AI technologies can solve complex industry problems far more effectively than using individual models in isolation.What's next for BuildNova

Our next goal is to transform BuildNova into a production-ready platform by:

Supporting real-time collaboration across multiple users. Integrating IoT sensors and drones for automated site monitoring. Enhancing AI-powered predictive risk analysis. Automatically generating and updating digital twins of construction sites. Deploying on AR smart glasses for completely hands-free operation. Integrating with construction platforms like Autodesk Construction Cloud and Procore.

Built With

• ar
• docker
• fastapi
• foundationdb
• ifcopenshell
• llamaindex
• opencv
• postgresql
• python
• qdrant
• qwen2.5-vl
• react.js
• ultralytics
• unity
• yolov11