• 

• 

  Cockcpit

Inspiration

Every closed-loop brain-computer interface decoder drifts over time — the signal-to-intent mapping degrades as the neural interface settles and the underlying signal changes. In a normal production ML system, that gets fixed with a silent redeploy. In a BCI, the "model" is wired into someone's nervous system, and a silent bad update isn't a UX regression, it's a safety issue. I wanted to build a system that took ML observability as seriously as any production ML pipeline — using DataHub as the actual source of truth for lineage, incidents, and decisions — while making the one place automation has to stop, human approval, completely non-negotiable.

What it does

SynaptoFlow watches a simulated BCI decoder for drift, using real angle-error and KL- divergence telemetry against a calibration baseline. When it crosses a guardrail, it raises a real DataHub Incident. Two agents wake up on their own: one drafts a plain-English diagnosis grounded in the decoder's real DataHub entity and lineage graph; the other runs a genuine recursive least squares regression, processed trial-by-trial, to propose a re-calibration for each channel. A clinician reviews both in a Streamlit cockpit — and only after they explicitly approve, edit, or reject does anything get written back: the incident resolves, the deployment's tags flip, and a Decision document is saved to DataHub, linked to the model and dataset — a durable, searchable record any future agent or clinician can look up, even though today's pipeline doesn't yet automatically pull past decisions back in as input to a new one.

How i built it

Four pieces, each mapping to a real DataHub capability rather than a parallel system: monitoring (drift detection + real Incidents, Tags, Structured Properties), diagnosis + calibration (MCP Server reads for context, a real RLS regression for the math), review (a human-gated Streamlit cockpit that's a pure file reader — zero live calls, zero recomputation on interaction), and deployment (incident resolution, tag updates, and a real Document write- back). Two lightweight watcher processes react automatically to new incidents and new approvals, so the only manual actions anywhere in the system are triggering a new patient session and making the clinical call. Every DataHub read and write goes through the real MCP Server. We developed entirely in GitHub Codespaces, partly out of necessity — working on a lower-spec laptop with limited RAM and storage meant local Docker-heavy development wasn't really an option — and that constraint ended up shaping the project for the better: it forced a genuinely reproducible setup from day one, which we then validated by testing the entire pipeline from a completely fresh GitHub account and a brand-new Codespace, catching real setup gaps a judge would otherwise have hit first.

Challenges i ran into

• MLModelDeployment entities have a confirmed gap across the MCP Server's get_entities, the default search, and the lineage graph — traced directly into the MCP Server's own source (entity_details.gql has explicit query fragments for every sibling ML entity type except this one). Worked around it with direct REST calls.
• save_document's related_assets field silently rejected an mlModelDeployment URN with a 400 error. Traced it to DataHub's RelatedAsset.pdl schema, which has an explicit entity-type allowlist that mlModel is on and mlModelDeployment isn't.
• datahub docker quickstart ships with token-based auth disabled by default, and the UI's error message ("contact your DataHub administrator") isn't actionable for a single-user local quickstart — required hand-editing the generated compose file.
• The MCP Server subprocess is launched via uvx; a fresh environment without uv installed fails with a bare FileNotFoundError, no hint that uv was the missing piece.

Accomplishments that i am proud of

A complete, four-way-verified real deployment against a live DataHub instance — not a staged screenshot: the deployment agent's own success line, tags confirmed via a direct REST call, a real Decision document correctly linked to the model and dataset, and calibration_params.json changed to the exact fitted values. I also built genuine crash-safety into the deployment agent — each side effect (incident resolution, tag update, document save, calibration update) is check pointed individually, so a failure partway through never double-resolves an incident or double-writes a document on retry, verified by deliberately injecting a failure mid-sequence and confirming the retry picked up exactly where it stopped. And the regression itself is real: tested against a zero-drift patient, the Calibration Agent correctly reports near-zero "drift" — noise floor, not a false positive — the behavior you'd trust from an actual statistical fit and distrust from a hardcoded one.

What i learned

That a genuinely autonomous agent system needs the same idempotency discipline as any distributed system — several real bugs only surfaced once we tested against partial failures (a network error mid-deployment, a stale local pointer to an already-resolved incident), not just the happy path. And that "human in the loop" only means something if the UI structurally prevents the loop from being skipped, not just if the button says "Approve."

What's next for SynaptoFlow

A more advanced audit and traceability layer, closer to how a real regulated device change would need to be logged — building on the Decision documents the system already writes to DataHub, versioned and structured enough to resemble the kind of change history a real approval process would expect. I'd also like to contribute the MLModelDeployment gaps i found back upstream to DataHub itself, since they're real and reproducible. More broadly, i plan to keep developing SynaptoFlow as i continue exploring the intersection of engineering and medicine, with the long-term hope of one day validating an approach like this in a real clinical research setting, under real regulatory and safety review.

Built With

• acryl-datahub
• apache-2.0
• datahub
• datahub-documents
• datahub-incidents
• datahub-lineage
• datahub-structured-properties
• datahub-tags
• docker
• github-codespaces
• graphql
• groq
• llama-3.1
• mcp
• mermaid
• mlops
• model-context-protocol
• numpy
• pandas
• python
• python-dotenv
• recursive-least-squares
• rest-api
• streamlit