Ghost Protocol

Inspiration

Every engineering team has a Daniel, the principal engineer who's survived every outage and carries twenty years of judgment in their head. The problem was never that Daniel won't help. It's that there's only one of him, and he can't be in every room.

I didn't want to build an AI that confidently answers everything. I wanted the opposite: an agent that inherits a specific person's expertise and knows where that expertise ends and defers. The hard problem in an expertise agent isn't recall. It's restraint.

What it does

Ghost Protocol reconstructs a senior engineer's judgment into an AI that shows up when they can't.

• Reconstruct — Build a ghost from knowledge fragments: incidents,architectural calls, war stories. Each one raises the ghost's fidelity.
  • Converse with trust — It doesn't treat everyone the same. A stranger gets a short, guarded answer; a trusted teammate gets the real take — the scar from 2011, the direct recommendation. Trust changes what it tells you, not how much.
  • Know the boundary — Ask something outside its domain — headcount, vendor choice, an ML call — and it says "not my call" and stops. No guessing.
  • Monitor with Arize — Every response is traced to Arize AX and scored by a custom boundary-adherence eval: grounded, deferred, or overstepped. A live
  • Live persona control via MCP: The agent calls the Phoenix MCP server every turn to fetch the latest persona prompt — edit it in Phoenix and the tone shifts instantly, no redeploy.
  • Deployment: Containerized to Google Cloud Run.

What I learned

• Boundary adherence beats generic faithfulness. "Did you stay in your lane?" is a more useful and measurable question than "did you use the right source?"
• Trust tiers beat flat access. Encoding who's asking as relationship context made the ghost feel like a person, not a search box.
• Tool-calling reliability is a real design constraint, not a given.

Challenges I faced

• Non-deterministic tool calls. Gemini 2.5 Flash would skip the MCP fetch on simple turns. A per-turn directive fixed it and I learned tool-call probability drifts down as session history grows.
• MCP in the cloud. It worked locally but threw silent 401s on Cloud Run, shell quoting had baked literal quotes into an env var. I hardened the config so it can't recur.
• Memory limits. A Node MCP subprocess alongside the Python ADK runtime blew past Cloud Run's 512 MiB default, a lesson in sizing polyglot containers.
• Two products, one brand. Arize AX (tracing + evals) and Phoenix (MCP prompts) are separate services with separate keys and endpoints. Decoupling them cleanly was half the integration work.

Built With

• arize-ax
• arize-phoenix
• fastapi
• gemini-2.5-flash
• google-adk
• google-cloud-run
• react
• vertexai