Manifest Match

An AI crew that reads your app's code and tells you where your Play Store declarations don't match it — before Google does.

Comment

Inspiration

I shipped three solo apps to Google Play in the last month, all built with AI coding tools. The code was never the hard part. What stopped me every time was everything after it: a privacy policy, the Play Data Safety form, review risk, store copy — and getting them wrong has real consequences, not just paperwork ones.

Along the way I actually hit the failure modes this product now detects: I collected calendar data without a real consent flow, I under-declared data sharing on a Data Safety form because I added an ad SDK late and forgot to update it, and I shipped screenshots that violated Play's aspect-ratio rule. None of these were caught by any policy generator, because every policy generator I tried just asks you what your app does and writes down the answer. Nothing checks the code.

Google's own enforcement pattern acts on the gap between your declared behavior and your actual behavior — not on the behavior alone. So I built the thing that closes that gap: an agent crew that reads the code first, and treats your declarations as a claim to verify rather than a fact to transcribe.

What it does

You give Manifest Match a public repository URL and answer five short questions (app name, category, target age, contact email, whether it's paid). A crew of agents then works through the repository:

  1. Analyst clones the repo and parses the actual manifests — AndroidManifest.xml, build.gradle, pubspec.yaml, package.json — and records every permission, SDK, and outbound network destination it finds, each one tagged with the exact file and line it came from.
  2. That output becomes the FactSheet — a single structured record that every other agent reads and nothing overwrites.
  3. Legal, Reviewer, and Growth run concurrently against that one FactSheet: Legal drafts a privacy policy, terms, and Play Data Safety answers; Reviewer checks the app against a 65-rule catalog of real Play policy failure modes and cites the official policy for each one; Growth drafts store listing copy and keywords.
  4. PM runs last, and its only job is to compare what Legal/Reviewer/Growth wrote against what Analyst actually found — and flag the difference. If Legal's privacy policy says "no location data" but the FactSheet shows a location permission, that's not a soft suggestion, it's a contradiction, and it goes into a short list of things only a human can resolve.

You get back draft legal documents, a prioritized list of review risks with evidence, store copy, and — most importantly — a short "confirm this yourself" list instead of a wall of AI-generated text you have to fact-check line by line.

How we built it

  • Google ADK orchestrates the crew as SequentialAgent(ParallelAgent(Legal, Reviewer, Growth), PM), with Analyst running first to produce the FactSheet the rest of the graph depends on.
  • Gemini 3.5 Flash-Lite handles Analyst's high-volume reading; Gemini 3.7 Flash runs Legal, Reviewer, and PM at thinking: high and Growth at lower thinking, since 3.7 Flash's variable thinking budget stood in for a Pro-tier model that doesn't currently exist at Gemini 3.5+ (see Challenges).
  • Cloud Run Jobs executes the pipeline itself (--task-timeout 30m, since a real repository scan is a multi-minute job, not a request/response call), while a separate, lightweight Cloud Run Service serves the Next.js dashboard and does nothing but accept the job submission form.
  • Firestore holds job state, per-agent live status, and the final artifacts. The dashboard's browser client subscribes to Firestore directly — not through Cloud Run — so progress streams in real time with no polling and no timeout ceiling on how long the user can watch. Every collection carries a 30-day TTL.
  • The rule catalog (65 rules: 42 critical, 19 warning, 4 info) is a YAML data file evaluated deterministically in Python, not a prompt. Same repository in, same findings out, every time.
  • The browser can read but never write. Firestore rules deny create/update/delete outright for client requests and deny listing a job's document (list: false) — a job is only readable if you already hold its random UUID; nothing enumerates the collection. Every write goes through the Admin SDK from the web tier or the Cloud Run job, never from a browser.

The core technical decision: facts never pass through a model

The first version of the pipeline asked a model to read the scan output and produce the structured FactSheet. On a real-world repository, that lost roughly 95% of what was actually found — 32 parsed permissions became 2, 220 dependencies became 6, because the model was summarizing a long report instead of transcribing it.

The fix was to stop asking a model to do that job at all. The FactSheet is now assembled directly in Python from the parser output. confirmed means a parser read that exact line — it's not a model's paraphrase of what it thinks it read. Models only get involved where judgment is actually required: writing the legal language, explaining a review risk, drafting store copy, and cross-checking the results.

Challenges we ran into

  • There is no Pro-tier Gemini model at 3.5 or newer. We benchmarked this directly (scripts/smoke_models.py): gemini-3.5-pro returns 404 on every location, and gemini-3.1-pro-preview doesn't meet the hackathon's Gemini 3.5+ requirement. We used Gemini 3.7 Flash's variable thinking level (high for Legal/Reviewer/PM) as the substitute for the reasoning-heavy roles, which turned out to be both cheaper and sufficient.
  • Region availability. Every model we needed is only served from the global Vertex AI location; regional endpoints like us-central1 404 entirely. Firestore stays regional; only the model calls needed to move.
  • ADK doesn't allow output_schema and tools on the same agent, which is why Analyst is split into a scan step and a separate FactSheet-formatting step rather than one agent doing both.
  • Cloud Run Services get CPU-throttled after responding, so "respond immediately, keep working for 15 minutes in the background" isn't a viable pattern there — which is why the actual scan runs as a Cloud Run Job, not inside the web service.
  • The FactSheet-fidelity bug above (32→2 permissions) was the single biggest architectural correction in the project, and it's the reason the product's central claim — "the fact came from code, not from a model" — is actually true rather than aspirational.

Accomplishments that we're proud of

  • A verified end-to-end run against a real, unmodified public repository (firebase/flutterfire) that surfaced an actual mismatch pattern: a "no data sharing" declaration next to code that shares data externally — the exact shape of the incident that motivated this project in the first place.
  • A deliberately-broken sample app with four planted mismatches (precise location stored server-side, an ad SDK that forces a "shared" Data Safety answer, user data sent to a third-party AI endpoint, account creation with no deletion path) plus one unplanned one: an unused RECORD_AUDIO permission left in as scaffolding, which the PM agent flagged on its own without being told to look for it.
  • A rule engine that reports what it cannot verify as unverifiable, not as "passed" — an empty "not checked" section would be a false claim of full coverage, and static analysis of source code never has full coverage.
  • A FactSheet pipeline that loses zero facts between the parser and the structured record, verified by a dedicated smoke test (smoke_analyst.py) after finding — and fixing — the version that lost 95% of them.

What we learned

That the interesting part of "AI agents build your app's paperwork" isn't the writing — it's the checking. Any model can produce a plausible privacy policy. The value is in refusing to let that policy disagree with the code, and in being honest, in the UI itself, about which parts are a verified fact and which parts are a model's draft that a human still needs to read.

What's next

  • Extend the FactSheet parser beyond Android/Flutter to React Native and native iOS declarations (currently best-effort or out of scope).
  • A GitHub Action so this becomes part of a release pipeline, not a one-off check — reusing the "one check per release" framing already designed for the product's retention model.
  • Turn the 12 catalog rules currently marked "cannot be evaluated from code alone" into either better heuristics or an explicit, permanent unresolved state the PM surfaces every time, rather than trying to force a verdict.

Built With

Share this project:

Updates

Submission history