STEM | Devpost

Inspiration

It started with a 3am Slack message.

I was six weeks into a SaaS internship, working on a fintech product. A senior engineer had just merged a migration that dropped a nullable constraint on the users.phone column — completely valid change. But our shared staging database immediately started rejecting inserts from three other branches that hadn't updated their seeders yet.

By the time I woke up, two people had wasted their entire morning debugging what looked like application logic failures. It wasn't. It was a shared database doing what shared databases do: reflecting everyone's changes at once, for everyone, whether they wanted them or not.

The "fix" was a Notion doc titled "Staging DB Rules":

Don't run destructive migrations before 6pm
Tag @backend-team before schema changes
If you broke staging, you own it until it's fixed I've seen versions of that document at every company I've worked at since. It's a monument to a solved problem that nobody bothered to solve properly.

I built STEM to delete that document.

The second motivation was PII. During another internship, I found that our staging database was a month-old snapshot of production — real users, real emails, real phone numbers. A contractor had a dump of it on their laptop. That's not a hypothetical compliance incident. That's a compliance incident that hadn't been discovered yet.

STEM solves both problems in one pipeline: isolated branches so no two PRs ever share state, and automatic PII anonymization so no developer ever sees real user data.

What It Does

STEM is a GitHub App + AWS backend that gives every pull request its own isolated Aurora PostgreSQL clone, anonymized and ready in under 30 seconds.

The flow:

PR opened
  → GitHub webhook fires (HMAC SHA-256 verified)
  → Aurora copy-on-write clone created (~28 seconds, zero data copied)
  → Writer instance provisioned on the clone
  → PII columns detected and anonymized (emails, names, phones, SSNs, IPs)
  → DATABASE_URL injected into Vercel preview environment
  → PR comment posted with connection details + anonymization report
  → Developer's preview environment hits an isolated, anonymized database

PR closed
  → Clone destroyed, env var removed, zero lingering cost

What developers get:

Full production-shaped schema on every PR — no more "works on staging" failures
Anonymized data by default — QA and contractors never touch real PII
Zero coordination overhead — no staging DB rules, no migration schedules, no "who broke it"
Automatic cleanup — no orphaned test databases accumulating cost The live dashboard shows real-time pipeline stages, provision times, PII columns masked per branch, and fleet cost per day. At 2 active branches, daily cost is $0.22.

How We Built It

Architecture

┌─────────────┐     webhook      ┌──────────────────────────┐
│   GitHub    │ ───────────────► │   STEM Backend (Vercel)  │
│  PR opened  │                  │                          │
└─────────────┘                  │  1. Verify HMAC sig      │
                                 │  2. Write DSQL metadata  │
                                 │  3. RestoreDBCluster     │──► Aurora DSQL
                                 │     (copy-on-write)      │    (metadata)
                                 │  4. Spin writer instance │
                                 │  5. PII anonymize        │──► Aurora PG Clone
                                 │  6. Inject DATABASE_URL  │    (isolated)
                                 │  7. Post PR comment      │
                                 └──────────────────────────┘
                                           │ polls
                                 ┌─────────▼──────────┐
                                 │  STEM Dashboard     │
                                 │  (Next.js / Vercel) │
                                 │  GitHub OAuth auth  │
                                 └────────────────────┘

The Clone: Aurora Copy-on-Write

The core of STEM is a single AWS API call that most engineers don't know exists:

await rds.send(new RestoreDBClusterToPointInTimeCommand({
  DBClusterIdentifier: `stem-pr-${prNumber}-${Date.now()}`,
  SourceDBClusterIdentifier: process.env.AURORA_SOURCE_CLUSTER_ID,
  RestoreType: 'copy-on-write',    // ← this is the magic
  UseLatestRestorableTime: true,
}))

Aurora's log-structured storage means a clone shares the same underlying storage pages as the source and only diverges on writes. A 500GB database clones in 28 seconds. The storage engine does the work — the API just needs one call.

The Metadata Layer: Aurora DSQL

Every branch has a state machine tracked in Aurora DSQL:

CLONING → PROVISIONING → ACTIVE → DESTROYED

DSQL is genuinely serverless — it scales to zero between PR bursts, handles concurrent writes from simultaneous webhooks without contention, and keeps the control plane cost near zero when teams aren't actively opening PRs. It's the right metadata store for an event-driven pipeline.

PII Anonymization: Zero Configuration, Fail-Closed

STEM's anonymizer works without any schema config and fails closed — if it can't sanitize the database, it never hands out the clone.

Discover — queries information_schema.columns to enumerate every text-type column across all tables. No hardcoded schema. Works on any database.
Detect — matches column names against layered PII patterns: high-confidence exact rules (email, ssn, tax_id, credit_card) plus regex patterns for semantic matches (*_phone*, *_address*, *first_name*, *ip_addr*, *secret*, *dob*, etc.). Bias is toward over-masking — fake data in a throwaway clone is harmless; an unmasked column is a breach.
Verify — pulls 10 row samples per candidate column and validates with regex to confirm actual PII values exist (not just PII-sounding column names with no data).
Mask — mass UPDATE with deterministic fake values using a seeded generator. Same real value → same fake value, so foreign key joins still work across tables.
Fail closed — if the schema scan matches nothing on a populated database, STEM moves the branch to masking_failed state. No connection string is ever injected. No env var is ever set. The PR gets a comment: "masking config needed." The clone is never exposed.
Audit — writes the masked column list to DSQL so the dashboard can show it, and flags any masking_failed branches with a red alert card. ### The Security Model: Cross-Account IAM

STEM uses the SaaS-standard cross-account trust pattern. No customer credentials are ever stored.

Customers run one command in AWS CloudShell:

curl -fsSL https://stem-frontend-six.vercel.app/api/aws/template -o /tmp/stem-role.json \
&& aws cloudformation deploy \
  --stack-name stem-access \
  --template-file /tmp/stem-role.json \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides ExternalId=stem-<your-id>

This creates an IAM role in their account that trusts STEM's control plane:

{
  "Principal": { "AWS": "arn:aws:iam::STEM_ACCOUNT:root" },
  "Action": "sts:AssumeRole",
  "Condition": { "StringEquals": { "sts:ExternalId": "stem-<hmac-per-user>" }}
}

The ExternalId is HMAC-SHA256 derived from the user's GitHub subject ID — unguessable, stable, scoped per user. STEM's role policy is scoped to arn:aws:rds:*:*:cluster:stem-pr-* for destructive actions — it literally cannot delete anything it didn't create.

The Frontend

Next.js 15 App Router, GitHub App OAuth with AES-256-GCM sealed sessions (Edge-compatible, no Node crypto module dependency), real-time dashboard polling DSQL, Tailwind + Recharts for visualization.

Challenges We Ran Into

Aurora DSQL Is Not Vanilla Postgres

DSQL doesn't support SERIAL/BIGSERIAL. No sequences. Auto-increment means gen_random_uuid(). Foreign key behavior differs. And in serverless contexts, connection pooling doesn't work — every invocation needs a fresh client and explicit teardown in finally blocks. I spent more time reading DSQL constraint docs than writing any single feature.

The Silent `anonymized_columns` Bug — and the Deeper Problem Behind It

My updateBranchState function had a subtle conditional:

// only write anonymized_columns when we also have the Vercel env var ID
if (extra?.vercelEnvId !== undefined) { ... }

When the Vercel API returned a response where id was undefined (silently, under certain conditions), the condition failed and anonymized_columns was never written to DSQL. The dashboard showed 0 COLUMNS ANONYMIZED for every branch. Two hours to find. Fix: COALESCE($2, vercel_env_id) and always write both fields unconditionally.

But that was the symptom. The deeper bug: the original anonymizer only handled a hardcoded users/payments schema. Any real database with different table names got zero columns masked — and the clone went live anyway, labeled safe. For a tool whose entire pitch is PII-safe clones, that's not a bug. It's the opposite of the product.

The rebuild queries information_schema to discover every text column dynamically, and adds a masking_failed state that holds the branch if the scan matches nothing on a populated database. STEM now fails closed: you cannot get a clone out of STEM that it couldn't sanitize. That single invariant is what makes the compliance claim real.

Multi-PR Vercel Env Var Conflict

The Vercel API returns 400 ENV_ALREADY_EXISTS if you try to create DATABASE_URL when it already exists — which happens the moment you have two PRs open simultaneously. The fix was an upsert: POST first, catch the 400, list existing env vars, find by key, PATCH. The catch: this edge case only manifests with two concurrent PRs, which requires the entire pipeline to be working. Found it in production testing, not in isolated unit tests.

Edge Runtime Cryptography

Next.js middleware runs in the Edge runtime — no Node crypto module. Session tokens are AES-256-GCM encrypted. I had to write the full seal/open implementation using only crypto.subtle (Web Crypto API), which works across both Edge and Node runtimes. IV prepended to ciphertext, Base64url encoding without Buffer, key derivation via SHA-256 digest. Standard cryptography, just without the ergonomics.

GitHub OAuth Redirect URI Mismatch

GitHub's "Be careful! The redirect_uri is not associated with this application" error hits when the OAuth callback URL doesn't exactly match what's registered. The root cause: VERCEL_URL changes per deployment (it's the deployment-specific URL), so every redeploy was sending a different redirect_uri. Fix: set APP_BASE_URL to the canonical production URL in env vars so the callback is always stable.

Accomplishments That We're Proud Of

28-second clone time regardless of database size. The copy-on-write restore is genuinely fast — not fast-for-a-database, but fast. Faster than most npm install runs.

Zero-config PII anonymization. No YAML files. No column lists. No schema declarations. STEM finds the PII itself, masks it, and tells you what it did. Connecting a new project doesn't require a data engineer.

$0.22/day for 2 active branches. Aurora Serverless v2 ACU-based billing means idle clones cost almost nothing. The economics work at every team size.

End-to-end working pipeline on a Vercel Hobby plan. The entire backend — webhook processing, Aurora clone orchestration, PII anonymization, Vercel API calls — runs as serverless functions. No dedicated infrastructure, no VMs, no orchestration layer.

A real security model, not demo security. Cross-account IAM with HMAC ExternalId, AES-256-GCM sessions, HMAC-verified webhooks, least-privilege role policies scoped to stem-pr-* resource names. Production-grade security on a hackathon timeline.

What We Learned

Copy-on-write is underutilized at the database layer. The pattern is everywhere in systems — container layers, filesystems, Git itself — but most developers don't realize Aurora exposes it directly via API. RestoreType: copy-on-write is one of the most powerful and least-discussed features in the RDS documentation.

The control plane is the product. The clone is one API call. The actual product is everything around it: state machine, idempotency, failure recovery, auth, observability. The hard engineering isn't the AWS call — it's building something that handles concurrent mutations, partial failures, and cross-account trust correctly at every step.

Serverless metadata stores change the economics of SaaS tools. Aurora DSQL scaling to zero means STEM has no baseline infrastructure cost. Every dollar of spend is directly proportional to customer activity. For a developer tool with bursty, event-driven usage patterns, that's the right cost model.

Edge-compatible crypto is more constrained than it looks. The Web Crypto API is powerful but verbose. Things that are one-liners in Node (crypto.createHmac, Buffer.from) require multi-step async operations with explicit key imports, IV management, and typed array manipulation. Writing a full AES-GCM seal/open implementation that works identically in Edge and Node runtimes was the most unexpectedly educational part of this project.

What's Next for STEM

Multi-tenant AWS fan-out — The architecture already supports it. The cross-account role assumption is implemented. The next step is wiring the webhook handler to look up the PR opener's connected AWS account and provision into their cluster. Each customer's clones live in their own account, billed to them, zero shared infrastructure.

Anonymization policies via .stem.yml — Let teams define custom column patterns, masking strategies (hash, fake, redact), and exemptions per table. Merge the config from the repo on every clone.

Branch lifecycle management — TTL-based auto-destroy for stale branches, configurable per org. Notifications before destruction. Emergency extensions for long-running features.

Org-level quotas and observability — Cap concurrent branches per org with a lease queue. Per-PR cost attribution. Spend alerts.

GitHub Actions native integration — Surface the branch DATABASE_URL as a step output so it works without Vercel, with any CI provider.

STEM is live at stem-frontend-six.vercel.app Built for H0: Hack the Zero Stack — Vercel x AWS Databases If your team has a "Staging DB Rules" Notion doc, STEM is for you.