Migration Release Acceptance

• 

  A successful migration can still leave the old application unable to read or write.

• 

  Independent application versions exercise disposable PostgreSQL and produce inspectable evidence.

• 

  A compatible expansion preserves the behavior required by both application versions in the demonstrated workload.

• 

  Restart the old application against the migrated schema and verify acknowledged values.

• 

  Local development, holdout, and regression outcomes, with scope limitations retained.

Inspiration

A database migration can succeed while the release still breaks users.

Consider a column rename: the SQL runs, the new application passes its tests, but an old instance still queries the original column. Rolling back the application does not restore that column either.

We built Migration Release Acceptance for backend developers and release reviewers who need evidence about this gap: can the declared old and new application workflows still read and write correctly against the migrated database?

What it does

The tool runs a reviewed migration and application workload against disposable PostgreSQL. Sixteen checks cover baseline behavior, migration execution, old-version and new-version reads and writes, mixed-version create/update sequences, and application rollback.

For rollback, it stops both application versions, restarts the old version against the migrated schema, and verifies the latest acknowledged values. It does not reverse the database migration.

Each run produces HTML, JSON, and Markdown evidence, with three explicit outcomes: GO_WITHIN_SCOPE, BLOCK, or INCONCLUSIVE. GitLab CI can use these outcomes as a release gate.

A manually mentioned GitLab Duo custom flow reviews CI evidence, checks its relationship to the merge request, and posts a bounded explanation. The executable checks determine the verdict; Duo explains the evidence.

How we built it

We built the MIT-licensed Python CLI with Psycopg, disposable PostgreSQL, independent HTTP application processes, and GitLab CI/CD.

A project integration declares trusted baseline and candidate SQL, application startup, and HTTP-operation mappings. We tested a pinned public FastAPI backend through its actual login and item CRUD routes.

We also challenged execution boundaries using public PostgreSQL and open-source migration cases. Sources, expected behavior, initial failures, and retest evidence were retained separately.

The GitHub Pages demo presents saved reports. Running database and application tests requires the CLI or CI.

Challenges we ran into

The hardest problem was avoiding false confidence. A successful migration is not proof of application compatibility, and a green harness self-test is not approval of a release candidate.

Rollback required exact value verification: a nonempty value can still be stale or wrong. We added checks against acknowledged writes, including legitimate empty strings.

External cases exposed transaction assumptions. PostgreSQL concurrent index operations require execution outside a transaction, so we added an explicit, reviewed nontransactional candidate mode while retaining the transactional default.

AI review introduced another boundary: stale pipeline evidence must not be presented as verification of a newer commit.

Accomplishments that we're proud of

• Reproducing a migration that succeeds while old-application behavior fails.
• Exercising a real FastAPI backend alongside synthetic reference fixtures.
• Completing actual GitLab Duo evidence reviews on recorded historical commits.
• Keeping incomplete runs explicit instead of treating missing evidence as a pass.
• In the latest local validation cycle, improving development results from 3/4 to 4/4 expected outcomes, then matching 4/4 fresh holdout outcomes and 63/63 retained regression expectations.

These are bounded test results, not an accuracy estimate or production guarantee.

What we learned

Useful automation needs a precise contract. Our checks establish what happened to the configured requests and values; they cannot establish safety for untested workflows.

We also learned to separate execution from explanation. Deterministic evidence makes a release decision auditable, while AI helps reviewers understand it without inventing a broader guarantee.

What's next for Migration Release Acceptance

Our next priority is independent developer trials: measure integration effort, repeated usefulness, and whether the evidence changes release decisions.

We also want fresh tests for timeouts and partial-effect recovery.

The current harness requires application-specific integration and uses sequential interleavings. It does not validate production-scale locking, concurrent traffic, replication, or performance. Overall capability remains partial within these boundaries.

Built With

• agent
• docker
• postgresql
• psycopg
• python