ECHO: Voice AI Assistant for Field Service Workers

Field workers tap, speak, and get clean professional service notes instantly. ECHO uses Amazon Nova Sonic for real-time voice AI and Nova Lite for smart summarization — no typing, no friction.

Comment

Inspiration

Field service workers spend hours every day doing paperwork that has nothing to do with their actual job. A pest control technician finishes a job, sits in a hot truck, and types out a service report from memory — trying to remember every detail, every treatment, every observation. That documentation ends up incomplete, inconsistent, and late.

We built ECHO because the people who do the hardest physical work shouldn't also be burdened with the hardest administrative work. What if a technician could just talk, and the paperwork wrote itself?

What It Does

ECHO is a hands-free voice AI assistant for field service workers. A technician speaks naturally about what they did on a job — treatments applied, conditions found, areas serviced, customer notes — and Amazon Nova Sonic responds in real-time with a conversational voice, confirming details, answering questions, and logging everything to the CRM automatically.

No typing. No hands. No end-of-day paperwork pile. Just talk and work.

Core features:

  • Real-time bidirectional voice conversation (not just transcription)
  • Amazon Nova Sonic sub-second response latency
  • Live tool calling mid-conversation: CRM queries, EPA chemical label lookups, weather checks
  • Works for pest control, HVAC, plumbing, electrical, and inspections
  • Designed for hands-busy environments: crawlspaces, ladders, attics

How We Built It

ECHO is a web application with a Node.js backend and browser-based voice capture using bidirectional streaming.

The audio pipeline captures PCM 16-bit 16kHz voice input in the browser via the Web Audio API, streams it over WebSocket to a Node.js server, which relays it to Amazon Nova Sonic through AWS Bedrock's bidirectional streaming API. Nova processes the voice stream and responds with voice output at 24kHz — creating a true back-and-forth conversation, not just transcription.

Tech stack:

  • HTML/CSS/JavaScript (frontend)
  • Node.js + Socket.IO (backend WebSocket server)
  • Amazon Nova Sonic (voice AI via AWS Bedrock bidirectional streaming)
  • Web Audio API (browser-side audio capture and playback)
  • Tool use: FieldWork CRM integration, LabelSDS.com EPA chemical lookups

Challenges We Ran Into

The hardest problem was real-time tool calling mid-conversation. ECHO doesn't just transcribe — it queries external systems while the user is still talking and weaves the results into its spoken response.

We discovered Nova Sonic has a tool count limit. With 9 tools registered, it wouldn't call any of them. Consolidating to 3 tools fixed it immediately. The lesson: keep tool count LOW for voice models.

We also had to solve bidirectional audio streaming — managing audio chunk byte alignment (16-bit PCM requires even byte counts), keeping the WebSocket open for Nova's voice responses, and handling natural turn-taking between user speech and Nova output.

Accomplishments We're Proud Of

  • Sub-second voice response latency — feels like talking to a human
  • Tool calling works mid-conversation (CRM lookup, chemical safety data, weather)
  • Zero hands required — built for workers in crawlspaces, on ladders, under dashboards
  • First voice AI assistant purpose-built for skilled trades workers

What We Learned

Amazon Nova Sonic performs best when given a domain persona, not just instructions. Framing the system prompt as "you are Jarvis, a field service AI assistant" rather than "respond to queries" produced dramatically more natural conversation flow.

The WebSocket event handshake order is critical — Nova expects initializeConnection → promptStart → systemPrompt → audioStart → audioReady in sequence. Getting that protocol right unlocked true bidirectional streaming.

Tool consolidation matters. Voice models need simpler tool interfaces than text models.

What's Next

  • ECHO is already in pilot testing with a real pest control company in Northwest Florida.
  • AR glasses integration (VITURE Beast) for true hands-free operation
  • Native iOS and Android apps
  • Direct CRM integrations (ServiceTitan, Housecall Pro)
  • Multi-language support for Spanish-speaking field workers
  • Fleet-wide deployment through FieldMatrix.ai

Built With

Share this project:

Updates