Cluvark FAQ Research Agent

• 

  Project Logo

Inspiration

Cluvark's buyer-facing assistant logs when it fails to answer — but never what was asked. Buyer questions are likely personal information under Australian privacy law, so we deliberately never store them.

That was the right call, and it left us blind. We knew the assistant was failing and had no way to learn what buyers actually needed. This agent recovers that information without touching a single real conversation.

What it does

Seven autonomous stages — Frame, Discover, Extract, Normalize, Cluster, Gap-analyse, Draft — research public sources and build a cited FAQ corpus for property buyers. No chat loop anywhere: an operator sets scope and budget, everything after runs unattended.

What makes it unusual is what it declines to do:

• It won't read Revenue NSW, the authoritative stamp duty source, because its licence permits reproduction only unaltered and only for non-commercial use. We're commercial, and every draft is an adaptation.
• It won't give advice. Property advice is regulated under the NSW Property and Stock Agents Act, so "what should I offer?" routes to a handoff taxonomy and goes to the human agent.
• It won't invent. When cited material doesn't support an answer, the drafter returns CANNOT\_ANSWER and the cluster produces nothing.
• It won't guess. Questions it can't confidently classify escalate for human review rather than being force-fit into the nearest category.

A human approves before anything reaches a buyer, writing to an append-only audit trail.

How we built it

Google ADK 2.7 SequentialAgent over seven stages, deployed as a Cloud Run Job in australia-southeast1. Gemini 3.5 Flash for canonicalisation and drafting on the global endpoint; text-embedding-004 regional for clustering. ADK sessions in Cloud SQL Postgres, secrets in Secret Manager, VPC egress restricted to private ranges.

The enforcement is structural, not prompted:

• The licence gate is a before\_tool\_callback that blocks the tool call — not a prompt asking the model to behave.
• Prohibited source tiers are unrepresentable: a CHECK constraint means they fail at insert, not at code review.
• Tier A and Tier B extraction return different Python types, and the Tier B type has no field capable of holding answer prose.
• Fetching respects robots.txt fail-closed, with SSRF guards, an origin allowlist and per-origin rate limiting.

130 offline tests — zero network calls, zero model calls in the default suite.

Challenges we ran into

The model swap silently truncated everything. Moving to Gemini 3.5 Flash, drafts came back as 4-token fragments. The SDK's convenience text accessor returns only the first content part, which on a thinking model can be reasoning rather than the answer. Every guard passed — the fragments were under the token ceiling, had valid citations, raised no flags — because nothing checked for a minimum.

Correct refusals and silent failures looked identical. An early run reported 13 drafts from a 36-item worklist and "0 refusals," because the counter only tracked fetch-policy blocks, not drafting outcomes. The pipeline was behaving correctly and reporting incoherently.

Our fetcher would invent URLs. Passing an intent with no seed revealed a code path that synthesised plausible slugs and tried them — thirteen variations on strata against a government domain. Every other gate in the system fails closed; this one improvised. We deleted it and made unseeded intents refuse at Frame.

Two correct changes composed into a wrong system. We widened intent matching to check the full registry, and separately tightened it to require keyword corroboration. Each was right. Together they escalated 19 of 37 clusters, because a keyword vocabulary written for one purpose became load-bearing for another.

Accomplishments that we're proud of

Licence verification found that three of seven government publishers we assumed were open are not. NSW Land Registry Services looks like a government source but is a private consortium operating the land titles registry under a 35-year concession — non-commercial use only. ASIC's MoneySmart is default-closed, granting Creative Commons over ten named document classes rather than general web content. The ATO uses a bespoke open licence that isn't Creative Commons at all, so a CC-only classifier fails closed on a source that is genuinely open — we wrote a separate detector for it.

Licence conditions became validation rules. CC BY 4.0 requires indicating that changes were made; every draft is a summarisation and therefore an adaptation, so every citation must carry the modified-form attribution. legislation.nsw.gov.au mandates a download date. The ATO's no-endorsement condition constrains answer phrasing. All three are hard rejects in code.

The refusals are the product. An agent that declines on IP grounds, declines to give regulated advice, and declines to answer beyond its sources is more useful in this domain than one that maximises output.

What we learned

Government open-licensing is far less uniform than it looks. We assumed .gov.au meant CC BY 4.0. NSW agencies split between CC BY 4.0 and CC BY 3.0 Australia, some are all-rights-reserved, and a privatised registry on a .com.au domain sits outside every Crown licence policy.

"Just re-express the facts" is the wrong instinct for an LLM pipeline. Duty rates are facts and facts aren't copyrightable, so we could in principle read a closed source and restate the numbers. We didn't. A model drafting "in its own words" from protected expression is producing paraphrase, and because it sees that source on every run the risk is structural rather than occasional. We cite the open-licensed instrument instead.

Inherited rules need re-deriving, not copying. Our regional-endpoint constraint came from a sibling agent that processes buyer conversations, where cross-border disclosure rules genuinely apply. Applied to a pipeline touching only public content, it was cargo cult.

Instrumentation is a correctness concern. Until refusals were coded and staged, we couldn't tell discipline from failure — and neither could anyone reading the output.

What's next for Cluvark FAQ Research Agent

Immediate. Unblock the sources we can't currently reach: legislation.nsw.gov.au and ato.gov.au both fail at robots.txt, which our fail-closed rule correctly treats as "no permission." That costs us stamp duty, strata and settlement coverage — four of eleven intent areas.

Near-term. Build the real review UI, replacing the demo-operator CLI: WebAuthn auth with per-(corpus, jurisdiction) reviewer authorisation. Resolve seed curation ownership — a seed URL asserts a page is in-licence and on-topic, which is the same judgement class as approving a citation, so it belongs with the reviewer role.

Then. A Phase 2 embedding router consuming the clustered corpus, including the handoff clusters as its negative set. Threshold-triggered campaigns driven by production miss-code rates, closing the loop the manual process currently spans. Victoria as a second jurisdiction — cooling-off and vendor disclosure diverge materially from NSW, and a single national answer would be wrong in at least one state.

Deliberately deferred. The Cloudflare D1 transport is a single labelled swap point, not an oversight. We'd rather ship one honest decision than two rushed ones.

Built With

• claude
• gemini-3.5-flash
• google-adk
• python