pulse (A Consulting Project Early Warning System)

Early warning system monitoring consulting projects for compound distress signals, matching 60 historical cases, running a Risk vs Recovery agent debate, and alerting partners before margin erosion.

Comment

Pulse: A Consulting Project Early Warning System

Inspiration

Consulting projects rarely fail suddenly; they erode.

Burn rate accelerates. Delivery velocity declines. Client communication shortens. By the time leadership intervenes, margin is already lost and the relationship is strained.

The problem is not missing data. It is fragmentation.

Financial metrics, delivery performance, and client sentiment live in separate systems. No mechanism continuously correlates these signals as a compound trajectory across all live projects.

Pulse was built to transform passive observability into proactive, evidence-based intervention.

What it does

Pulse monitors live consulting projects for sustained, compound distress signals.

It detects when burn acceleration, delivery velocity drop, and negative client sentiment occur simultaneously over four consecutive weeks - not a spike, a trajectory.

When triggered, Pulse:

  1. Searches 60 historical consulting cases using vector similarity to find structurally comparable precedents
  2. Runs a structured Risk vs Recovery multi-agent debate, each agent arguing from retrieved evidence
  3. Produces a confidence-calibrated decision:
    • > 85% → Automatic action + FYI email to partner
    • 60–85% → Partner approval required, action blocked
    • < 60% → Escalation email, no automated action

Every decision, the full reasoning trail, and the partner's response are indexed back into Elasticsearch, creating an auditable, learnable system that improves with every intervention.

How we built it

Pulse is built on Elasticsearch, Kibana Agent Builder, and a Python orchestration layer.

Compound Signal Detection

An ES|QL query continuously evaluates all live projects against four thresholds simultaneously. It triggers only when distress is sustained - not a single anomaly, but a confirmed trajectory:

$$ D = (B_{\text{accel}} > \tau_1) \land (V_{\Delta} < \tau_2) \land (S_{\text{slope}} < \tau_3) $$

A separate time-series index stores four weeks of weekly snapshots per project. A second ES|QL tool computes week-over-week acceleration, so the Sensor Agent can report "burn rate increased from £48,200 to £71,800 per week over four consecutive weeks" rather than simply "burn rate is high."

Structural Similarity Search

Each historical project is encoded as an 8-dimensional distress fingerprint:

$$ \vec{x} = [ \text{burn acceleration},\ \text{velocity delta},\ \text{sentiment slope},\ \text{phase},\ \text{team size},\ \text{client tenure},\ \text{days to milestone},\ \text{margin at risk} ] $$

k-NN cosine similarity retrieves structurally comparable cases:

$$ \text{sim}(\vec{x}, \vec{y}) = \frac{\vec{x} \cdot \vec{y}}{|\vec{x}|\ |\vec{y}|} $$

The 60 historical projects are organised into six coherent scenario clusters by industry and distress pattern: fintech delivery scope creep, healthcare delivery team burnout, retail design sentiment collapse, and so on. This ensures that k-NN matches reflect genuinely comparable situations, not random synthetic noise.

Why a debate, not a single agent

The multi-agent debate is not cosmetic. A single agent given both the risk evidence and recovery evidence will anchor on whichever argument appears first in the context window. Separating them into adversarial roles with independent tool access forces the system to construct the strongest possible case for each position before the Arbiter weighs them.

The Arbiter is required to run an additional historical search before deciding if the Risk and Recovery agents disagree by more than 0.15 confidence points; it cannot average two positions, it must find additional evidence. That is the difference between a system that reasons and one that summarises.

Five-agent pipeline

  • Sensor Agent: compound signal detection + 4-week trend analysis
  • Diagnosis Agent: historical search + playbook retrieval
  • Risk Agent: argues for conservative intervention, cites specific failure cases
  • Recovery Agent: argues for internal resolution, cites specific rescue cases
  • Arbiter Agent: weighs debate, searches for additional evidence on disagreement, decides

A Python orchestrator chains all five agents via the Kibana Agent Builder API, managing retries, 300-second timeouts, and the confidence-calibrated action logic.

Challenges we ran into

The Kibana Agent Builder API required some discovery, the payload key is input not message, and the response structure differs from the standard Anthropic SDK. Multi-tool agent calls on large indices regularly exceeded 120 seconds, requiring retry logic and extended timeouts.

Designing the historical data was harder than the retrieval logic. Random synthetic data makes k-NN matches feel meaningless, the system returns neighbours that share no coherent story. The solution was designing six cluster archetypes where each project within a cluster shares a realistic distress pattern with controlled noise. The key insight was that "fintech delivery phase scope creep escalated" retrieves the right historical cluster more reliably than raw vector similarity on incoherent data, because the cluster labels themselves are the most discriminative features.

Getting the Arbiter to be genuinely evidence-driven rather than performing superficial arbitration required adding an explicit constraint: if agent confidence scores diverge by more than 0.15, it must call its search tool before deciding. Without that constraint, the Arbiter simply averaged the two positions.

Accomplishments that we're proud of

  • A compound trajectory detector that distinguishes sustained deterioration from isolated anomalies
  • k-NN similarity that retrieves structurally meaningful precedents because the underlying data was designed for coherence, not volume
  • A five-agent pipeline where the debate is enforced; each agent must argue from retrieved evidence, and the Arbiter must search before deciding when agents disagree
  • A confidence spectrum that maps directly to governance: auto-execute, draft for approval, or escalate with the threshold explicit and auditable
  • A full decision trail indexed back to Elasticsearch, so every recommendation and every human override becomes future signal

What we learned

Multi-dimensional correlation is meaningfully different from threshold alerting. Correlating three signals that are each individually noisy produces a compound signal with far lower false positive rates than any single signal alone.

Data coherence matters more than model complexity in similarity search. A well-structured 60-project dataset with coherent clusters outperforms a larger random dataset because the retrieval quality depends on what the neighbours actually have in common.

Multi-agent debate improves robustness only when grounded in enforced evidence retrieval. Without the mandatory tool call constraint, the agents produce confident-sounding summaries of their priors rather than arguments from data.

What's next for Pulse

  • Replace synthetic historical data with real anonymised consulting project data
  • Adaptive confidence calibration based on historical decision accuracy where the system learns which of its confidence scores were right
  • Expand signal inputs: resource churn, scope change frequency, milestone slippage rate
  • Direct integration with consulting workflow tools so approved actions trigger automatically in project management systems

Built With

Share this project:

Updates