Foundry — human-controlled support agents from any website

Foundry crawls a business website, finds the gaps and contradictions its support bot can't answer, and routes them to the owner. The bot answers only from human-approved policy — never a guess.

Comment

Inspiration

Most support bots fail silently. They answer confidently from whatever they scraped, and nobody ever sees the questions their knowledge base cannot answer. For a small business — a solar installer, a clinic, a workshop — half the real knowledge lives on the website and half in the owner's head. We wanted an agent that discovers that missing half autonomously, and then does the one thing most bots refuse to do: ask the human, and wait.

What it does

Give Foundry a website. It:

  1. Crawls the site and drafts a support knowledge base (in the demo: 43 entries from 6 pages).
  2. Ingests policy documents (PDF) and detects gaps and contradictions between the document and the live site — e.g. a 2024 policy promising a binding fixed-price offer from photos, while the site promises only an indicative range.
  3. Routes every open question to the owner in a human-in-the-loop queue. The owner answers once, in their own words.
  4. Publishes with full version control: the answer becomes ACTIVE, the previous entry is marked SUPERSEDED and kept for history. Nothing is silently overwritten.
  5. Serves a support bot that answers only from ACTIVE, human-approved entries. No answer in the base means a declared gap — never a guess.
  6. Attacks itself before launch: six adversarial personas (confused, angry, returns edge case, off-topic, prompt injection, contradiction prober) probe the bot while a separate evaluator scores every answer. Off-topic requests refused correctly, prompt injection contained, zero hallucinations. Requests that knowledge cannot solve are reported as capability gaps, not turned into fake FAQs.
  7. Runs a nightly loop unattended: Cloud Scheduler → Pub/Sub → Cloud Run Job re-reads the site and files anything new for the owner's approval. Eleven consecutive nightly runs in the demo project.

Autonomous discovery. Human-controlled policy.

How we built it

Everything runs on Google Cloud, in europe-west1:

  • Google ADK orchestrates the agent pipeline: crawler, interviewer (gap & contradiction detection), adversarial personas, evaluator, and the support bot.
  • Gemini 3.5 Flash via Vertex AI (google-genai SDK) powers extraction, detection and evaluation.
  • Cloud Run hosts the Foundry service (stateless) and the fictional demo site, LumoCasa Energy.
  • Firestore (Native) is the single source of state: tenants, entries, questions, and the DRAFT → ACTIVE → SUPERSEDED lifecycle with soft deletes.
  • Cloud Scheduler + Pub/Sub + Cloud Run Jobs drive the nightly review through a push subscription to Foundry's /api/pubsub/night-shift endpoint.
  • Gemma 2 (2b-it) was deployed and verified on a dedicated Vertex AI endpoint for question classification.

Multi-tenant by design: each tenant is an isolated knowledge base, created on first use.

Challenges we ran into

Our biggest lesson: silent failures, not crashes, are the real risk in agentic systems. Four times during the build a feature looked functional — green tests, clean logs — and was inert in practice: a textarea destroyed by polling, a field dropped between modules, a publish action unreachable from the UI. None of it showed up in tests. All of it showed up the moment a human clicked through the real interface.

That lesson shaped the product itself. Foundry's whole premise is that an agent should surface what it cannot do — gaps, contradictions, capability limits — instead of papering over them. We ended up building the tool we needed while building it.

What we learned

  • Human-in-the-loop is not a fallback; it is the architecture. The moment the owner becomes the only writer of truth, hallucination stops being a model problem and becomes a policy problem — one you can actually enforce.
  • Refusals are a feature. An evaluator that scores "I don't have that information yet" as correct behavior changes what you optimize for.
  • Demo determinism beats ambition. Every claim in our video is something we can reproduce on demand.

What's next for Foundry

Wiring the Gemma classifier into the live pipeline, surfacing the nightly job's findings as a first-class owner review page, and turning the adversarial loop into a continuous readiness score a business owner can trust before going live.

Built With

Share this project:

Updates

Submission history