Helixstreet — autonomous genomic fleet

Drop raw DNA in a bucket and walk away: five Gemini+Gemma agents screen PII, call variants with DeepVariant, flag drug-gene risks, and file an auditable digital twin. No chat. No clicks. No human.

Comment

Inspiration

Every genomic sample that reaches a lab starts a relay of humans: compliance review, PII scrubbing, pipeline babysitting, annotation lookups, record-keeping — days of coordination per file. We wanted to know what it looks like when the relay is replaced by a governed fleet of agents that the file itself wakes up. Not a chatbot with tools — a system with no chat loop anywhere.

What it does

Drop a raw sequencing file (BAM) into a Cloud Storage bucket and walk away:

  1. Intake agent screens the patient manifest with Model Armor (prompt-injection, malicious-URI, PII filters) and de-identifies it with Cloud DLP.
  2. Variant-calling agent dispatches Google DeepVariant as an asynchronous Cloud Batch job, checkpoints state to a Firestore Memory Bank — and the fleet goes dormant.
  3. Minutes later the VCF lands; Eventarc wakes the fleet again. The analyst agent parses the real calls and cross-references 1000 Genomes in BigQuery.
  4. The pharmacist agent screens variants against a curated CPIC drug-gene reference (e.g. "CYP2C19 poor metabolizer — clopidogrel may not activate") and attaches peer-reviewed papers it fetches live from Europe PMC.
  5. The steward agent persists a digital twin behind a hash-chained audit ledger whose integrity the dashboard re-verifies on every refresh.

Measured result: raw reads to fully-annotated twin in ~9 minutes, zero human touches — and three samples processed concurrently by the same fleet. Every reasoning step is an OpenTelemetry trace in Cloud Trace.

How we built it

Four principles shaped the architecture:

  • Right-sized hybrid model tier. The high-throughput intake gatekeeper runs on Gemma 4 31B (Gemini API backend, native function calling; graceful fallback to Gemini if the key is absent), while the four reasoning agents run on Gemini 3.5 Flash. At population scale, screening every sample with a frontier model is waste; a small open model is the production answer.
  • Events decide when, the model decides how. Routing between pipelines is deterministic (Eventarc + a regex router); behavior inside each stage is agentic — Gemini 3.5 Flash (Vertex AI, global endpoint) driving tools through Google's Agent Development Kit (Agent, SequentialAgent, Runner).
  • Dormancy over polling. The async gap while DeepVariant runs is bridged by Firestore state, not a waiting process. The same mechanism would carry a multi-week job.
  • Identity at the platform layer. Three least-privilege service accounts: the analyst can read BigQuery but cannot launch compute; the dashboard reads state but cannot invoke agents; the fleet endpoint accepts exactly one identity — Eventarc's. Zero API keys in the codebase — the single key (Gemma's Gemini-API key) lives in Secret Manager.
  • Governance as data. Agent registry in Firestore, audit entries written by the agent that did the work, hash-chained so tampering breaks the chain.

Stack: Gemini 3.5 Flash · ADK · DeepVariant · Cloud Run · Cloud Batch · Eventarc · Cloud Storage · Firestore · BigQuery · Cloud DLP · Model Armor · Cloud Trace · FastAPI · Streamlit.

Challenges we ran into

Five production incidents, each of which hardened the system:

  1. Model placement — gemini-3.5-flash serves only from Vertex's global endpoint, while Batch jobs are regional; one env var was doing two jobs.
  2. The idempotency trap — we claimed events as processed before success, so a failed run swallowed its own Eventarc retry. Claims now release on failure: exactly-once on success, self-healing on error (later proven live when a quota-exhausted run failed, was redelivered, and completed untouched).
  3. GCS-FUSE's vanishing directories — empty directories have no backing object and evaporate when the kernel cache expires, killing DeepVariant's final write minutes into the job. We back the output prefix with a placeholder object before dispatch.
  4. Quota-aware agents — concurrent pipelines cannibalized a thin per-minute model budget; model-level 429 backoff (15s→120s) plus right-sized concurrency fixed it.
  5. Unpinned dependencies — an upstream release broke Cloud Build mid-hackathon; all dependencies are now pinned from a verified environment.

Accomplishments that we're proud of

  • A genuinely autonomous multi-agent system: upload in, governed digital twin out.
  • Real heavy compute (DeepVariant on Batch), real enterprise guardrails (Model Armor + DLP), real population genomics (1000 Genomes), real clinical references (CPIC).
  • The analyst agent noticed — unprompted — that VCF and BigQuery coordinates disagree by exactly one base pair (1-based vs 0-based) and reasoned through it. A join-by-position ETL would have silently returned zero matches.
  • 3 concurrent patients → 3/3 twins, 6/6 pharmacogenomic findings each, zero errors.

What we learned

Autonomy is an infrastructure property before it is a model property: exactly-once event handling, claim release on failure, durable state across dormancy, and platform-level identity are what let an agent be trusted to act alone. The model supplies judgment inside stages; the architecture supplies the right to act.

What's next for Helixstreet — autonomous genomic fleet

Whole-genome scale on larger Batch shapes, per-department registries with A2A hand-offs, dbSNP joins in BigQuery to assign rsIDs to called variants, and managed Vertex AI Memory Bank for cross-session personalization.

Built With

  • bigquery
  • cloud-batch
  • cloud-dlp
  • cloud-run
  • cloud-storage
  • cloud-trace
  • deepvariant
  • docker
  • europe-pmc
  • eventarc
  • fastapi
  • firestore
  • gemini
  • gemma
  • genomics
  • google-adk
  • google-cloud
  • model-armor
  • opentelemetry
  • pharmacogenomics
  • python
  • secret-manager
  • streamlit
  • vertex-ai
Share this project:

Updates

Submission history