Harbourmaster Tenders

Governed multi-agent tender review: five specialist agents analyze contracts, humans approve high-risk steps, and every call is traced in Phoenix with Elastic precedent search.

Comment

Inspiration

Enterprises want to use LLMs for procurement and contract review, but most pilots fail for predictable reasons: uploaded tenders can carry prompt injection, generic models do not know internal policy, multi-agent flows are opaque, and legal teams cannot sign off on fully autonomous output.

We built Harbourmaster Tenders as a governed control plane for that gap—not another chatbot, but a multi-step tender review workflow where agents do the heavy lifting and humans retain authority when it matters.

The name reflects the role: a harbourmaster directs traffic, enforces rules, and decides what may enter safely.

What it does

Harbourmaster Tenders is a workflow governance control plane for agentic procurement review. A reviewer uploads a tender and the system:

  • Runs an inline guard on input (ALLOW · HUMAN_REVIEW · DENY)
  • Segments the document into ordered clauses with stable IDs
  • Retrieves similar policies and precedents from Elasticsearch
  • Deploys five specialist agents in parallel (legal, financial, delivery, IP/data, compliance)
  • Aggregates weighted risk and runs a verifier revision loop
  • Routes to auto-continue or pauses for human review when risk is high
  • Negotiates counter-clauses and drafts a reviewer-ready summary
  • Persists review artifacts back into Elastic for future searches

Every model call is traced in Arize Phoenix. A Governance Copilot answers questions over guard telemetry, saved reviews, Phoenix experiments, and Elastic precedent search via native tools plus Phoenix MCP and Elastic MCP.

The Streamlit UI includes a live execution timeline that streams LangGraph phases and per-specialist progress while a review runs.

How we built it

Harbourmaster Tenders is a Streamlit reviewer UI on top of a LangGraph state machine:

  1. Guard evaluates tender input
  2. Segmenter splits the document into stable clause IDs
  3. Elastic retrieves similar policy/precedent snippets
  4. Five specialist agents run in parallel
  5. Aggregator + verifier merge findings and request revisions when needed
  6. Governance router auto-continues or escalates to human review (interrupt/resume)
  7. Negotiator + drafter produce counter-clauses and a reviewer-ready summary
  8. Artifacts persist back into Elastic

Every model call goes through a GovernedLLM wrapper to Gemini (OpenAI-compatible API) and emits OpenInference spans to Arize Phoenix.

We ship Docker Compose (ui + self-hosted phoenix + elastic) with env-driven local/cloud modes, a Configuration page, make config-check validation, and VPS-ready deployment. Progress visualization uses graph.stream() with custom events from each workflow node.

Challenges we ran into

  • Making governance visible in real time. Early demos used a single spinner while five agents ran in parallel. We switched to LangGraph streaming with custom events so reviewers see guard verdicts, specialist completion, verifier loops, and HITL routing as they happen.
  • Partner integrations that complement each other. Phoenix answers what happened and was it safe; Elastic answers what precedent and policy apply. Wiring both into the copilot without fragile MCP-only paths required a hybrid native + MCP tool design.
  • Deployment realism. Docker service URLs, browser console URLs, and Phoenix project IDs behave differently on localhost vs VPS. We split server-side URLs (phoenix:6006) from browser URLs (PHOENIX_CONSOLE_URL) and resolved Phoenix console links by project ID.
  • Balancing autonomy and control. Too many automatic approvals erode trust; pausing every clause erodes velocity. A weighted risk threshold plus explicit reviewer decisions gave us a practical middle ground.
  • Configuration sprawl. Per-service modes (local Phoenix, cloud Phoenix, local Elastic, cloud Elastic) needed a clear precedence model: defaults → optional UI overrides → environment variables.

Accomplishments that we're proud of

  • A real multi-step business workflow, not a single-shot prompt—segmentation, parallel specialists, verifier loop, governance routing, negotiation, and draft generation.
  • Two-layer governance that separates security verdicts from commercial risk scoring.
  • Meaningful partner depth: Phoenix traces, governance dashboard, red-team experiments, Phoenix MCP; Elastic indexing, precedent retrieval, Elastic MCP.
  • Human-in-the-loop that actually interrupts the graph—not a cosmetic approval button after the fact.
  • Live processing visualization showing internal workflow state while agents execute.
  • Deployable stack with local and cloud profiles, configuration UI, and a working demo on sample high-risk tenders.

What we learned

  • Security and business risk are different problems. A prompt injection is not the same as a high-risk indemnity clause—they need different mechanisms and telemetry.
  • Observability is not optional for agents. Without traces, you cannot explain why a specialist flagged clause C3 or defend a decision to procurement/legal.
  • Memory makes agents enterprise-ready. Elastic precedent search grounds findings in indexed policies and prior reviews while preserving clause-level evidence quotes.
  • Human-in-the-loop must be first-class. LangGraph interrupt/resume lets reviewers approve, reject, and annotate before counter-clauses and summaries are generated.
  • MCP works best as an accelerator, not the foundation. Native tools keep demos reliable; MCP extends the copilot when Phoenix/Elastic subprocesses are available.

What's next for Harbourmaster Tenders

  • Richer retrieval: hybrid semantic + keyword search with reviewer feedback loops into Elastic
  • Policy versioning: tie clause findings to explicit policy revision history
  • Phoenix eval automation: gate releases on red-team regression scores from experiment runs
  • Enterprise auth: SSO-backed reviewer identities and audit exports for compliance teams
  • PDF ingestion improvements: layout-aware clause extraction for scanned tenders
  • Hosted demo: stable public deployment with Phoenix Cloud and Elastic Cloud profiles

Built With

Share this project:

Updates