RecallOps

RecallOps uses DataHub context, PostgreSQL agentic memory, and validated AI investigation to turn data alerts into safe, human-approved next steps.

Comment

Inspiration

Data incidents are rarely solved by the alert alone. Responders need to understand the affected asset, ownership, downstream impact, what happened in similar incidents, and which assumptions still need to be verified before anyone acts.

RecallOps was built to make that investigation traceable. It combines live catalog context from DataHub with durable incident memory, an evidence-gated AI investigation, and a human decision gate.

What it does

RecallOps turns a data-quality alert into a reviewable incident investigation.

  • Reads source, ownership, schema, and lineage context from DataHub through MCP.
  • Retrieves comparable resolved incidents from PostgreSQL-backed agentic memory.
  • Shows a match delta: shared context, changed context, and assumptions that must not transfer automatically.
  • Runs an AI investigation that produces structured competing hypotheses and a bounded recommendation.
  • Validates every selected hypothesis and evidence reference against the server-side evidence bundle.
  • Keeps remediation simulation-only until a human operator explicitly records a decision.
  • Preserves the investigation, reviewer challenge, decision, and outcome as an auditable replay.

How we built it

RecallOps is a React and TypeScript application running on Vinext.

DataHub OSS runs locally and is accessed through the DataHub MCP server. A narrow, loopback-only MCP bridge verifies that the available toolset is read-only before the application uses it for live catalog context.

Incident dossiers, historical resolutions, decisions, and audit state are stored in local PostgreSQL with pgvector enabled. PostgREST provides a loopback-only bridge for the worker-compatible application runtime.

For model-backed investigation, RecallOps uses GPT-4.1 mini through AIMLAPI. The model receives an immutable server-side evidence bundle and returns structured output. Zod validation and server-side checks reject unsupported hypotheses or evidence IDs before results are displayed.

Docker Desktop starts the local DataHub and PostgreSQL services. The repository includes reproducible setup, smoke tests, reset instructions, and a demo runbook.

Challenges we ran into

The hardest part was making “agentic” useful without making it unsafe or overstating what the system knows.

A historical incident can be relevant without being a valid answer, so we built the visible match delta and reviewer constraints rather than automatically applying past resolutions.

We also wanted live DataHub context to be meaningful. The runtime path now uses MCP, not only a separate smoke test, and it is deliberately limited to metadata reads.

Finally, model output is not treated as truth. RecallOps forces the investigation to work from a bounded evidence bundle, validates its citations, and keeps the proposed action simulated until a person approves it.

Accomplishments that we're proud of

  • A live, read-only DataHub MCP integration in the application runtime.
  • Durable PostgreSQL agentic memory with comparable-incident retrieval and a visible match delta.
  • A real model-backed investigation with evidence validation rather than a free-form chat response.
  • A structured investigator, historian, planner, and reviewer flow that makes reasoning inspectable.
  • A human approval gate, idempotent decisions, and an audit replay.
  • A fully local Docker-based environment that can be reproduced without hosted infrastructure.

What we learned

We learned that the valuable part of an incident-response agent is not an autonomous action. It is the disciplined handoff between grounded context, remembered experience, model-assisted reasoning, explicit constraints, and accountable human judgment.

We also learned that MCP is most useful when it is a real runtime boundary with clear capabilities—not merely a compatibility checkbox.

What's next for RecallOps

Next, we would add more data-platform connectors, richer semantic retrieval for historical memory, configurable remediation policies, and production-grade authentication and deployment controls.

The core direction stays the same: preserve context, show uncertainty, validate evidence, and let people remain responsible for consequential actions.

Built With

Share this project:

Updates