ContractorPilot — Autonomous Voice Copilot for Contractors

Autonomous voice agent that calls suppliers & trade subs to negotiate prices, verify live warehouse stock, and turn jobsite walkthroughs into signed client proposals in minutes.

Comment

Inspiration

Every residential and commercial renovation starts with a jobsite walkthrough: measuring rooms, assessing demolition needs, and jotting down materials. But the moment a general contractor steps off the jobsite, they hit the "Contractor Phone Bottleneck":

  • Spending 10+ hours every week playing phone tag with wholesale building supply reps to check live stock, negotiate contractor trade discounts, and verify freight delivery dates.
  • Calling multiple trade subcontractors (licensed plumbers, master electricians, finish painters, tile setters) to negotiate daily rates and calendar availability.
  • Taking 3 to 7 business days to compile all this scattered data into a proposal — often losing the contract to a competitor or mispricing materials due to volatile prices.

I asked myself: What if an autonomous voice agent could place those phone calls on behalf of the contractor while they drive between jobsites, negotiate trade pricing, verify warehouse inventory, and deliver a client-ready, signed proposal in 3 minutes?

That is why I built ContractorPilot.

What it does

ContractorPilot is an autonomous voice copilot that transforms a contractor's rough jobsite walkthrough into a signed client proposal:

  1. 🎙️ Voice Walkthrough Dictation: Contractors can speak naturally into their phone or browser (or load curated demo presets like The Miller Residence 1,300 sq ft luxury remodel). The system captures room dimensions, surface materials, and scope.
  2. 📋 Instant AI Scope Breakdown: Automatically segments the walkthrough into structured trade subcontractor labor tasks and a comprehensive bill of materials (e.g., Calacatta porcelain tiles, zero-VOC paint, recessed LED fixtures, brass valves).
  3. 📁 Custom Subcontractor & Supplier Directory (Rolodex): Contractors can import their private directory of preferred trades and suppliers via .csv, .xlsx, or .txt. CALL-E prioritizes dialing their trusted network before falling back to regional suppliers.
  4. 📞 Autonomous Voice Procurement with CALL-E: CALL-E directly telephones suppliers and subcontractors. It introduces the project, inquires about specific quantities, negotiates volume discounts, verifies live warehouse stock, and collects freight lead times.
  5. ⚖️ Multi-Criteria Weighted Bid Matrix: Normalizes incoming bids and calculates an intelligent weighted score to rank the best options: $$ \text{Score} = 0.40 \cdot S_{\text{price}} + 0.20 \cdot S_{\text{stock}} + 0.15 \cdot S_{\text{lead_time}} + 0.10 \cdot S_{\text{delivery}} + 0.10 \cdot S_{\text{reliability}} + 0.05 \cdot S_{\text{discount}} $$
  6. 📑 Proposal Studio with Real-Time Markup: The contractor adjusts their gross margin slider (e.g., 15% to 25%), and the platform instantly generates a professional, legally structured client proposal (Proposal N° CP-2026-001) complete with milestone payment draw schedules and client signature lines.

How I built it

  • CALL-E Voice Integration: Built using the official calle-ai Python SDK (v0.7.0). I engineered specialized task system prompts and structured JSON extraction schemas (client.calls.create_and_wait) to extract critical procurement metrics (unit pricing, trade discounts, pallet stock, delivery timelines).
  • Interactive Telephony Sandbox: To guarantee judges have a zero-friction evaluation experience (even without phone line provisioning or running out of call credits), I built an interactive sandbox featuring synthesized Web Audio dial tones, an HTML5 Canvas real-time audio waveform visualizer, 14 recorded voice tracks, and turn-by-turn dialogue transcripts.
  • Backend Architecture: High-performance asynchronous FastAPI (Python 3.10+) service with WebSockets streaming call status, real-time extraction cards, and bid comparison calculations.
  • Frontend & UX: High-contrast, luxury dark UI built with vanilla HTML5, modern CSS3 (glassmorphism, CSS variables, fluid responsive layouts), and vanilla JavaScript. No heavy frontend framework dependencies to ensure lightning-fast load times.
  • Deployment: Production Docker container, turnkey 1-click Render blueprint (render.yaml), and automated GitHub Actions CI/CD pipeline.

Challenges I ran into

  1. Structuring Conversational Voice Extraction: In phone conversations, building suppliers often respond colloquially (e.g., "Yeah, got about 15 boxes left on the pallet, probably 600 sq ft, could give you 5% off if you take the lot"). Designing robust CALL-E prompt instructions and JSON schemas that reliably extract normalized numerical prices, percentage discounts, and stock numbers required careful prompt calibration.
  2. Realistic Audio-Visual Feedback: Phone calls are fundamentally audio-driven. To make the dashboard feel alive, I engineered a canvas-based soundwave engine synchronized with synthetic DTMF tones and audio playback so users visually experience the call status in real-time.
  3. Flexible Rolodex Data Ingestion: Contractors store supplier contacts in varied spreadsheet formats. I implemented a resilient parser supporting CSV, XLSX, XLS, and TSV formats with auto-mapping headers.

Accomplishments that I'm proud of

  • True End-to-End Workflow: As a solo developer, I built an end-to-end commercial tool that bridges voice dictation $\to$ AI extraction $\to$ autonomous telephony negotiation $\to$ multi-criteria scoring $\to$ signed contract proposal.
  • Zero-Friction Evaluation: Anyone can clone the repository, run python run_server.py, and immediately test the entire workflow with full interactive audio and visual simulation.
  • Real Business Value: Tackling a genuine $10B+ pain point in residential construction, cutting estimate turnaround from 5 days down to under 3 minutes.

What I learned

  • The true power of CALL-E lies not just in conversing, but in its ability to execute goal-oriented business negotiations and return actionable structured JSON data from voice dialogues.
  • How critical conversational latency and natural phrasing are when speaking with trade subcontractors and busy warehouse clerks on the phone.

What's next for ContractorPilot

  • Automated Purchase Order Dispatch: Auto-sending digital POs to suppliers via email/SMS immediately upon proposal signing.
  • Live In-Call Fallback: Instant push notification alerting the contractor if a supplier cannot fulfill an order, allowing human takeover mid-call.
  • Multi-Language Support: Bilingual CALL-E voice agents in Spanish and English to seamlessly communicate with diverse trade teams on jobsites.
  • Mobile Native App: iOS and Android companion apps with camera photogrammetry for instant room scanning alongside voice dictation.

Built With

Share this project:

Updates

Submission history