FrontBox AI

Live on AWS: Bedrock triages, a Converse agent merges a duplicate, a person approves, Amazon Location ranks crews, and a test crew accepts with one WhatsApp tap via AWS End User Messaging Social.

Comment

FrontBox AI gets the right field crew on the job with one WhatsApp tap, and folds many reports into one incident.

For any organization that dispatches field crews, often subcontractors outside its systems. Each domain maps to a crew skill (CI-ONLY; live runs used the grid tenant):

  • Roadside assistance: BATTERY_JUMP or FLATBED_TOWING
  • Home assistance for insurers: EMERGENCY_PLUMBING
  • Utilities: HIGH_VOLTAGE_LINE
  • Medical advisory: TELE_DOCTOR

For judges

Proof labels. LIVE-CURRENT: the release judges test (gdFnZY2, released 2026-10-27 22:48 UTC). LIVE-OLD: an earlier runtime. CI-ONLY: hosted tests, fake providers.

What to look for, by judging criterion

  • Technical Execution (40%): three CDS services in code; a real WhatsApp offer, one tap, ASSIGNED; Bedrock, Titan, Step Functions and Amazon Location in one workflow. WhatsApp LIVE-CURRENT (two runs, one crew phone); SMS and SES CI-ONLY.
  • Potential Value (20%): crews with no app or account accept by WhatsApp, and many reports become one incident. On 150 synthetic reports, 0 electrical hazards were missed where keywords missed 29 (model plus deterministic rules, precision 0.724). LIVE-OLD, one run.
  • Demo (20%): the final run, report to Assigned, on the portal and the crew phone (the owner's, registered as Crew A). LIVE-CURRENT, n = 1.
  • Creativity (10%): the tenant policy decides where a person approves, and the approver sees model confidence, the enforcing layer and the reason codes apart. LIVE-CURRENT.
  • Functionality (10%): owner-approved release, canary, 22 of 22 presence probes (infrastructure only); 2,810 JavaScript/TypeScript, 916 Python and 48 Playwright tests. Release LIVE-CURRENT; tests CI-ONLY.
  • Best WhatsApp Solution: a crew with no app or account accepts with one tap. LIVE-CURRENT, worker side.

Expected result of your own run: Escalated, unassigned. Fail-closed by design: no approved template and no open 24-hour window means nothing is sent (observed once, LIVE-CURRENT). The video shows the run to Assigned.

Inspiration

Dispatchers merge duplicate reports by hand and phone crews one by one. A storm multiplies both.

The hardest case: Nikolaos, stuck in his car in Marousi, Athens, under a live 20kV line. It looks like roadside, but he needs a grid line crew, not a tow truck.

Illustrative showcase scenario, not a measurement: a tap after 43 seconds instead of a 47-minute phone tree, and 15 reports folded into 1 incident (fixture timestamps, a narrative and a seeded cluster).

What it does

The final live run, n = 1:

  1. Report. A Commander enters the report with a GPS pin. LIVE-CURRENT
  2. Triage. Bedrock (one forced tool call) and deterministic safety rules set P1, an electrical hazard and HIGH_VOLTAGE_LINE. LIVE-CURRENT
  3. Correlate. A bounded Converse agent merges a reworded report (similarity 0.61): one dispatch. LIVE-CURRENT
  4. Decide. A versioned tenant policy: allow, require approval, require dual control, escalate or deny; all audited. REQUIRE_APPROVAL LIVE-CURRENT; the others CI-ONLY
  5. Offer. Amazon Location ranks the skilled crews; WhatsApp Accept and Decline buttons follow. LIVE-CURRENT
  6. Resume. One tap resumes Step Functions: Assigned. LIVE-CURRENT, two runs

Fallbacks. Decline, timeout or a failed receipt tries the next crew (up to 5 offers), then Escalated, unassigned (CI-ONLY).

Optional, insurer view. Policyholders call their insurer, not the company that sends the crew. Partner page demo in the console is that entry point: a fictional insurer page with the FrontBox report widget embedded (deployed; not in a recorded run).

How we use WhatsApp

WhatsApp is the crew's main interface (detail: Best WhatsApp Solution field).

Live (LIVE-CURRENT, two runs, one crew phone)

  • Zero-seat crew. No app, no account; interactive buttons inside the crew's own 24-hour window.
  • Out and back. SendWhatsAppMessage sends offer and progress buttons; taps return via SNS, SQS and Lambda to SendTaskSuccess.
  • Opaque IDs. Buttons carry only ACCEPT_<id> or DECLINE_<id>; the task token stays in DynamoDB.

Built and tested (CI-ONLY)

  • Template fallback. Outside the window, an approved template (none yet); with neither, nothing is sent.
  • Consent. STOP and START checked before every send.
  • Delivery ladder. A failed receipt moves the offer on; an unanswered offer can get one SMS nudge; one answer wins.

How we built it

  • CDS, WhatsApp: AWS End User Messaging Social. LIVE-CURRENT
  • CDS, SMS and email: AWS End User Messaging SMS (two-way, STOP and START) and Amazon SES v2. CI-ONLY, nothing sent yet
  • AI: Bedrock triage, a Converse agent (at most 4 turns and 8 tool calls), Titan Text Embeddings v2. LIVE-CURRENT
  • AI safety: a Bedrock guardrail. Request shapes LIVE-OLD
  • Media: Rekognition masks faces and plates; Transcribe reads voice notes. LIVE-OLD
  • Matching: the Amazon Location route matrix. LIVE-CURRENT
  • Workflow: Step Functions task tokens, Lambda, EventBridge, SNS, SQS. LIVE-CURRENT
  • Data: DynamoDB outbox and ledger, S3 evidence, KMS encryption. LIVE-CURRENT
  • Access and web: Cognito roles, API Gateway (HTTP and WebSocket), CloudFront with a Certificate Manager certificate. LIVE-CURRENT
  • Release and operations: CDK and CloudFormation, CodePipeline with an owner approval, CodeBuild, a CodeDeploy canary, CloudWatch alarms and metrics, Secrets Manager for webhook secrets. LIVE-CURRENT

25 AWS services in all, wired together in one CDK app.

Challenges we ran into

  • API versions. AWS rejected WhatsApp v26.0, v25.0 and v24.0 on 2026-10-27 (LIVE-OLD); this release uses v23.0 (LIVE-CURRENT).
  • The hour boundary. Approval at 20:59, offer at 21:00: the safety fingerprint changed and a second approval opened (LIVE-OLD). Now the hour counts only when a rule reads it (12 new tests, CI-ONLY).
  • Honest numbers. The first correlation run scored recall 0.162 and F1 0.279; tuning on the same pairs reached 0.892, so it is in-sample (LIVE-OLD).

Accomplishments that we're proud of

  • Offer on the crew phone: about 6 s after Approve. LIVE-CURRENT, n = 1
  • ASSIGNED: about 69 s after Approve, mostly the crew reply pending. LIVE-CURRENT, n = 1
  • Report to acceptance: 144 s on the final run; median 101 s over two. LIVE-CURRENT, n = 1 and n = 2: a record, not a benchmark
  • Domain and severity accuracy: 0.847 and 0.700 (keywords: 0.553 and 0.420). LIVE-OLD, one run, 150 synthetic reports
  • Duplicate correlation: precision 1.00, recall 0.892, F1 0.943. LIVE-OLD, in-sample, 60 synthetic pairs

What we learned

  • Provider rules show up only live.
  • A safety fingerprint must hold only the facts a rule reads.
  • Deployment approval, business approval and crew acceptance are separate gates.

What's next

  • An approved offer template for crews outside the window.
  • A real benchmark: median offer-to-accept over at least 10 live acceptances.
  • More live proof: citizen WhatsApp, SES production access, held-out evaluation, load test.
  • Amazon Bedrock AgentCore: move the bounded correlation agent onto AgentCore.

Pre-existing work. FrontBox is an existing product brand with a legacy product. This repository is a new implementation built during the submission period. No code from the legacy product is incorporated; third-party open-source packages are listed in package-lock.json.

AWS CDS Agentic AI Hackathon -Sept. 2026.

Built With

Share this project:

Updates

Submission history