RigSense — AI That Prevents Accidents Before They Happen
Inspiration
Construction sites are fast, dangerous, and overloaded with information—yet the people doing the physical work operate with almost no real-time support.
Rigging and crane-related failures consistently rank among the leading causes of struck-by and caught-in construction incidents (OSHA/BLS data). Since rigging is involved in nearly every crane lift across thousands of active job sites every day, even small errors can have catastrophic consequences.
The problem isn't that safety knowledge doesn't exist—it's that it's trapped in binders, manuals, and PDFs that a rigger can't access while both hands are occupied.
Workers routinely make split-second sling-angle and load-capacity judgments based on experience alone, often in noisy, fast-paced environments where there's no time to consult a load chart.
When we realized Meta Smart Glasses place both a camera and a private audio channel directly at the worker's point of attention, the idea became clear:
What if AI could see exactly what the worker sees—and speak up before an unsafe lift becomes an accident?
That's how RigSense was born.
What it does
RigSense is an AI-powered rigging and lift safety copilot built around three core capabilities.
🏗️ Live Lift Validation
RigSense estimates sling angles using known rigging hardware dimensions as scale references.
It then evaluates the lift using the engineering relationship:
It then evaluates the lift using the engineering relationship:
$$C_{\text{effective}} = C_{\text{nominal}} \times \sin(\theta)$$
where:
- \(C_{\text{nominal}}\) = Rated rigging capacity
- \(\theta\) = Sling angle measured from the horizontal
When the effective lifting capacity becomes lower than the suspended load, RigSense immediately alerts the worker with contextual voice feedback.
Example:
"Sling angle is 42 degrees. Load capacity reduced by 30 percent."
🔒 Hands-Free Lockout/Tagout (LOTO)
When a worker looks at unfamiliar equipment, RigSense identifies it and reads the correct lockout-tagout procedure aloud, step by step, eliminating the need to stop work and search through documentation.
📋 Automated Near-Miss Logging
Every detected hazard is automatically logged with structured metadata, helping organizations capture incidents that are traditionally underreported due to slow and manual reporting workflows.
How we built it
RigSense uses a two-tier architecture because a safety alert that arrives late isn't a safety alert.
Edge Tier
Runs locally on wearable hardware and handles:
- Camera frame capture
- Lightweight object detection
- Immediate hazard triggers
- Local text-to-speech synthesis
Target latency:
< 80 ms
AI Reasoning Tier
Open-source Vision Language Models perform higher-level reasoning including:
- Sling angle estimation
- Geometric verification
- Context-aware safety analysis
- Engineering rule validation
Models:
- Qwen-VL
- LLaVA-1.6
Action Arbiter
Before notifying the worker, RigSense:
- Deduplicates repeated alerts
- Ranks hazard severity
- Reduces unnecessary interruptions
- Logs every safety event automatically
Tech Stack
| Layer | Technology |
|---|---|
| Computer Vision | YOLO-World, YOLOv8 |
| Vision-Language Models | Qwen-VL, LLaVA-1.6 |
| Speech Recognition | Whisper |
| Text-to-Speech | Piper |
| Backend | FastAPI |
| Database | Supabase, PostgreSQL |
| Architecture | Edge + Cloud Hybrid |
Why Smart Glasses?
We deliberately chose smart glasses instead of fixed-site cameras.
The challenge isn't simply detecting hazards somewhere on the construction site.
The challenge is delivering the right information to one specific worker exactly when they can still prevent the accident.
Smart glasses combine:
- First-person vision
- Hands-free interaction
- Private audio feedback
- Immediate contextual assistance
into a single wearable device.
Fixed cameras can identify hazards, but they cannot provide personalized, real-time guidance without requiring a worker to reach for another device.
Our AI pipeline was intentionally designed to be camera-agnostic.
We validated the reasoning pipeline using standard camera feeds, allowing future integration with Meta's Wearables Device Access Toolkit without redesigning the underlying AI.
Challenges we ran into
Monocular Angle Estimation
Estimating real-world sling angles from a single RGB camera is difficult without depth information.
Instead of relying on stereo vision, we used known rigging hardware dimensions as scale references to approximate real-world geometry.
Avoiding Alarm Fatigue
An AI safety system that interrupts workers too frequently eventually gets ignored.
Rather than alerting on every detection, RigSense treats interruption itself as a design decision by ranking hazards based on severity and suppressing redundant notifications.
Balancing Latency with AI Reasoning
Large Vision-Language Models provide accurate contextual reasoning but are too slow for immediate hazard detection.
Splitting inference between edge detection and cloud reasoning gave us both fast reactions and deeper understanding.
Hardware Availability
Meta Smart Glasses hardware and SDK access were not fully available during development.
Instead of overclaiming hardware integration, we validated the AI pipeline using conventional camera input while designing the system for seamless migration to Meta's Wearables Device Access Toolkit.
What we learned
Building RigSense taught us that industrial AI is about much more than object detection.
The real challenge lies in deciding when an AI should interrupt someone performing dangerous physical work.
That's a human-factors problem as much as an AI problem.
We also learned that wearable-first AI creates capabilities impossible for site-wide camera systems.
First-person perspective combined with private audio feedback fundamentally changes how safety assistance can be delivered.
Finally, separating what has been experimentally validated from what is planned made both our engineering process and our pitch significantly stronger.
What's next
Our roadmap includes:
- ✅ Native Meta Smart Glasses integration using the Wearables Device Access Toolkit
- ✅ Ground-truth validation with professional rigging crews
- ✅ Improved monocular 3D angle estimation
- ✅ Expanded hazard detection beyond rigging
- ✅ Exclusion-zone monitoring
- ✅ Equipment-specific procedural guidance
- ✅ OSHA compliance reporting
- ✅ Enterprise safety dashboards
- ✅ Predictive lift-risk analytics
Impact
RigSense transforms wearable AI into an active industrial safety partner.
Instead of analyzing accidents after they happen, RigSense helps prevent them before they occur—providing workers with real-time guidance exactly when and where it matters most.
Every crane lift deserves an AI safety copilot.
Built With
- ai
- computer
- construction
- fastapi
- glasses
- industrial
- language
- llava
- models
- multimodal
- operations
- piper
- postgresql
- python
- qwen-vl
- rigging
- safety
- smart
- supabase
- technology
- tts
- vision
- whisper
- yolo-world
- yolov8
Log in or sign up for Devpost to join the conversation.