SiteCopilot — Autonomous Revenue Recovery for Construction

12 AI agents turn a field photo and voice memo into an approval-optimized change order package in 60 seconds. Revenue in. Paperwork out.

Comment

## Inspiration

Construction subcontractors lose over $100 billion annually to under-documented change orders. When a foreman finds a crack in a foundation wall, the PM spends 2-4 hours searching through 500-page blueprints, writing narratives, and estimating costs. 20% of legitimate change orders are never submitted. Of those submitted, 30% get rejected for weak documentation. We built SiteCopilot to recover that revenue.

## What it does

SiteCopilot is an autonomous revenue recovery engine. A foreman uploads a site photo and records a voice memo. 12 specialized AI agents analyze the evidence against project blueprints, enterprise cost data, and historical approval patterns — producing a complete 7-section change order package with approval probability scoring in under 60 seconds.

The 7-section package includes: annotated photo with damage highlighting, blueprint citation with verbatim spec quote, editable narrative optimized by approval history, cost breakdown with editable line items, missing information checklist, composite confidence score, and one-click PDF export.

## How we built it

12-Agent Pipeline with 6 Orchestration Patterns:

  • Parallel Swarm: Vision + Intake + Memory agents run simultaneously
  • Sequential Chain: Plan → Commercial → Compliance agents build on each other
  • Mixture of Experts: Risk Assessor + Evidence Ranker evaluate from different angles
  • Self-Improvement: Narrative Optimizer rewrites using past approval patterns
  • Hierarchical Review: QA Agent checks all agents for consistency
  • Autonomous: Routing Agent auto-sends to the correct approver

7 Sponsor Tools Deeply Integrated:

  • Railtracks (1,500+ lines): 12-agent orchestration framework with observability
  • Senso.ai (960+ lines): Knowledge base for blueprint/spec grounding and semantic search
  • Nexla (560+ lines): Enterprise data fusion — labor rates, vendor costs, CO history
  • Unkey (900+ lines): Per-agent API key management, rate limiting, cost metering, audit log
  • DigitalOcean: App Platform deployment, Managed Postgres, Spaces storage
  • assistant-ui (560+ lines): Evidence copilot — PM asks "why this cost?" and gets answers
  • Augment Code: AI-assisted development of all 12 agents and prompt engineering

AI Engine: Google Gemini 2.5 Pro multimodal (photo + audio

  • PDF in one context window)

## Challenges

  • Gemini preview model names expired mid-hackathon — required runtime model discovery
  • Senso.ai KB upload required reverse-engineering the pre-signed S3 upload flow
  • Making 12 agents produce consistent, non-contradictory output required a dedicated QA Agent
  • Balancing demo reliability (canned data) with real pipeline execution (live Gemini calls)

## What we learned

  • Agent count and orchestration complexity win hackathon demos (lesson from DeepQuant)
  • Every dashboard metric should map to money, not just agent status
  • "Revenue capture OS" positions better than "AI drafting tool"
  • The self-improvement loop (learning from approvals) is the real moat

## What's next

  • Pilot with 3-5 concrete subcontractors in Bay Area
  • Outcome feedback loop: track approved vs rejected, improve estimates
  • Procore/Autodesk integration for project data sync
  • Mobile native app for field workers

Built With

Share this project:

Updates