VoltHawk

Inspiration

US utilities operate 200,000+ miles of transmission lines graded D+ by ASCE. Manual inspectors review drone footage at 20 miles/day with a 17% error rate. Vegetation alone causes 38% of all outages. A single failure costs $1M-$100M+.

What it does

VoltHawk processes drone inspection footage through TwelveLabs Marengo 3.0 and Pegasus 1.2 on AWS Bedrock to detect insulator damage, vegetation encroachment, and structural corrosion. Each finding is scored against NERC FAC-003-4, FAC-501-3, and NESC Rule 215 compliance thresholds, cross-referenced against maintenance history, and presented in a dashboard with an AI co-pilot.

How we built it

• Pipeline: Video → S3 → Marengo 3.0 (512-dim embeddings) → cosine similarity search (12 detection queries) → Pegasus 1.2 (structured condition descriptions) → NERC severity scoring
• Dashboard: Next.js 14, Tailwind CSS, Recharts. Features: NERC compliance cards, severity distribution, expandable findings with editable severity and inspector notes, maintenance cross-reference with auto-escalation, AI co-pilot (Claude), CSV export and report generation
• Validation: 77.3% precision, 100% recall, 87.2% F1 on 17 ground truth labels from 20-min drone footage

Challenges

• Marengo cosine similarity scores for transmission footage fall in 0.10-0.25 range — required threshold calibration
• Pegasus structured JSON output was inconsistent — implemented streaming fallback
• 1.7GB video files require compression for API upload limits

What we learned

• Semantic video search eliminates the need for labeled training data — natural language queries replace custom object detection models
• Regulatory grounding (NERC) transforms a detection tool into a compliance tool
• Cross-referencing video findings with maintenance history creates intelligence neither source provides alone

Built With

• amazon-web-services
• anthropic-claude
• aws-bedrock
• jupyter
• next.js-14
• python
• recharts
• tailwind-css
• twelvelabs-marengo-3.0
• twelvelabs-pegasus-1.2