Inspiration
Construction is one of the world's most dangerous industries, yet the people performing the work still rely on paper manuals, printed drawings, and safety documents that are impossible to access when both hands are occupied. The information exists—but it rarely reaches workers at the moment they need it most.
The Physical AI challenge inspired us to rethink construction safety from the worker's perspective. Instead of expecting workers to stop and search for information, we asked a simple question:
What if AI could see exactly what a worker sees, understand what they're trying to accomplish, and speak up before a mistake becomes an accident?
That idea became ARGUS—an AI construction intelligence co-pilot built for Meta Smart Glasses that provides proactive, hands-free assistance in real time.
What it does
ARGUS transforms Meta Smart Glasses into an intelligent construction assistant that continuously understands the work environment and provides contextual voice guidance.
Key capabilities include:
Detects construction activities such as beam installation, rebar tying, conduit routing, and structural inspection. Identifies safety hazards including missing PPE, unsafe proximity to edges, suspended loads, and missing safety equipment. Verifies ongoing work against BIM models and construction drawings to detect missing or incorrectly installed components. Predicts potential hazards before they become accidents using continuous risk forecasting. Answers spoken questions using site-specific safety manuals, SOPs, and engineering documents through a Retrieval-Augmented Generation (RAG) system. Generates privacy-preserving incident logs for supervisors and insurers while avoiding continuous worker surveillance.
Instead of simply detecting objects, ARGUS understands construction intent and assists workers with proactive, spoken guidance in real time.
How we built it
ARGUS uses a hybrid edge-cloud architecture designed for wearable devices with limited compute and battery.
Our pipeline consists of:
Edge Vision Layer: A lightweight YOLO-based perception model performs object detection, PPE detection, and pose estimation on-device or on the paired smartphone. Task Intent Recognition: AI identifies the construction activity currently being performed instead of only recognizing objects. Risk Assessment Engine: Multiple perception signals are combined into a continuous risk score, inspired by autonomous vehicle perception systems. Blueprint & BIM Verification: The live scene is compared with engineering drawings and BIM models to identify construction mismatches. Vision-Language Reasoning: When higher-level reasoning is required, selected frames are sent to a multimodal AI model for contextual understanding. Site Memory (RAG): Safety manuals, SOPs, permits, and engineering documents are indexed so workers can ask natural language questions hands-free. Voice Copilot: AI delivers concise spoken guidance through Meta Smart Glasses without distracting workers from their tasks. Privacy-First Incident Logging: Only safety-critical events are stored, reducing bandwidth, protecting worker privacy, and maintaining compliance.
Challenges we ran into
Designing AI for construction is significantly more complex than conventional object detection.
Some of our biggest challenges included:
Balancing low-latency inference with the limited computational resources available on wearable hardware. Deciding which AI tasks should run on-device versus in the cloud to optimize speed, battery life, and reliability. Building a system that understands construction context rather than simply detecting objects. Designing a privacy-first architecture that avoids continuous surveillance while still creating useful incident records. Integrating multiple AI capabilities—including perception, reasoning, blueprint verification, and document retrieval—into a unified real-time workflow.
We addressed these challenges through a two-tier perception pipeline, selective cloud inference, event-triggered processing, and retrieval-grounded AI reasoning.
Accomplishments that we're proud of
We are proud that ARGUS goes far beyond traditional PPE detection systems.
Our key achievements include:
Built an AI system that understands what the worker is doing, not just what the camera sees. Combined computer vision, multimodal AI, RAG, and BIM verification into a single construction intelligence platform. Designed an edge-first architecture suitable for wearable devices with realistic latency constraints. Developed a privacy-preserving incident logging system instead of continuous recording. Created a solution that improves both worker safety and construction productivity. Delivered an AI assistant that aligns closely with the Physical AI vision by perceiving, reasoning, predicting, and assisting in the real world.
What we learned
Building ARGUS taught us that Physical AI is not just about recognizing objects—it is about understanding context.
We learned:
Construction safety depends more on timely decision support than on simply providing more information. Real-world AI systems require careful trade-offs between latency, accuracy, privacy, and battery efficiency. Retrieval-Augmented Generation is essential for providing reliable, site-specific answers instead of relying solely on general-purpose AI knowledge. Edge-cloud collaboration enables powerful AI experiences while remaining practical for wearable hardware. Designing for workers means prioritizing simplicity, hands-free interaction, and trust over complex user interfaces.
What's next for ARGUS
Next steps include:
Integrate directly with the Meta Smart Glasses SDK for live deployment. Expand hazard detection to cranes, electrical systems, confined spaces, and heavy equipment. Improve blueprint verification with precise BIM alignment and automatic progress tracking. Fine-tune predictive hazard forecasting using real construction site data. Pilot ARGUS with construction companies to validate safety improvements and productivity gains. Build supervisor dashboards with analytics, compliance reporting, and insurer integrations. Enable continuous learning across multiple construction sites to make ARGUS increasingly intelligent over time.
Our long-term vision is for ARGUS to become the AI construction co-pilot that every worker can rely on—helping teams build safer, smarter, and more efficiently through real-time physical intelligence.
Log in or sign up for Devpost to join the conversation.