Molly

Inspiration

Review is where a change gets understood, and the diff fights it. In a survey of 1,355 developers, "only 10.2% of reviewers consider alphabetical ordering optimal" and "more than half (57.6%) note that it increases context switching" (Rahman, Codabux and Roy, ICSE 2026). We wanted a big merge request to read as one connected change.

Understanding the code isn't the end of it. In a dbt project the code is only the recipe; what people read is the output. "72% of respondents now prioritize AI-assisted coding… while only 24% prioritize AI-assisted pipeline management, including testing and observability" (dbt Labs, State of Analytics Engineering 2026). That life after code is what we wanted to automate. The failures are real: England's COVID-19 dashboards missed 15,841 cases because of a data-load fault (Public Health England, 2020), and Facebook settled advertiser lawsuits over inflated video metrics for $40 million (Fortune, 2019).

What it does

Reviewing a big merge request today means going back and forth through one long diff. With Molly, it means looking at one connected change. Agents do the work; people approve outcomes.

The review lens. On every merge request, CI builds one page:

The diff developers already know: a file tree, Viewed marks, and unified or split views.
Links on the code itself: a name that code proved has a dotted underline. Hover it for a peek of what it reaches; click to unfold that code under the line.
Plan, Stack and Mine lenses: they filter the same diff.
Code decides: it splits the diff into changes, proves the links (calls, imports, dbt ref(), CI needs), places Semgrep and secret-detection findings on the changes they touch, and fails the job if any changed line isn't shown.
A GitLab Duo flow explains: groups by purpose, explanations, severities and a padding likelihood. CI checks its note against code's facts.
GitLab Orbit adds memory: callers outside the diff, each file's history, reverts and incidents, and a suggested reviewer who is never the author.

When a changed file was changed before by a merge request that was later reverted after an incident, code marks it history risk, high, puts the merge request in Draft and holds the release.

The lifecycle behind it, for dbt projects:

Data guard: CI builds the data twice and diffs it, through to the dashboards. On !13, Board Revenue for September −17.78%.
Owner decision: only a data owner who is not the author can approve. The guard refused the author's approval on !13 and recorded Bhawna's.
Release gate: holds packaging and deploys until each data change is decided.
Deploy: GitLab Pages and Google Cloud Run, through workload identity federation.
Production watch: checks production on every main pipeline and daily.
Incident and revert: on !8, which was merged anyway, the watch found 73 orders missing. A second Duo flow opened Incident #3 ("Was this predicted? Yes") and prepared the revert !9, which a person merged.

How we built it

Review lens (Python and one self-contained HTML page):
- diffmap splits the diff and checks coverage;
- links proves the links;
- lenses places the findings;
- memory reads GitLab Orbit and REST;
- history_risk turns proven reverts and incidents into the hold.
Deterministic core: data_diff, rehearse, data_watch, owner_decision, release_gate and handoff, with unit tests that gate the release.
Duo Agent Platform: three custom flows as code (review-lens, data-release-guard, data-incident), linted in CI and published to the AI Catalog.
GitLab CI/CD: both data builds, SAST and secret detection, a container image in the GitLab registry and container scanning, a GitLab Release per data release, GitLab Pages, a Cloud Run deploy and a daily schedule.
Data: a dbt project on DuckDB with an enforced contract and two dashboard exposures, each with its data owners. The data is synthetic and deterministic.
Project page: a feature tour rebuilt from scratch after Skiper UI's Apple Feature Block. Each feature opens a card with the run that proves it, while that feature's real recording plays beside it. Motion follows shadcn/ui and Skiper UI patterns.

Challenges we ran into

A review view developers don't have to learn. We dropped a node-and-line map and a step-by-step reading order: both were harder to use than the diff. We kept the familiar diff and put the links on the code.
A merge request bigger than the agent's read. !18 has 32,881 changed lines. Code builds and checks the whole structure; the agent writes for what it read, and says so.
Human approval inside a flow failed after approval (a known platform bug, gitlab-org/gitlab#632998), so we moved the decision onto the merge request.
No Maintainer rights. owner_decision turns the pipeline red until a non-author owner decides, and the release gate holds whatever gets merged anyway.
Flows can't start flows, so CI starts them through the Flows API.

Accomplishments that we're proud of

The review lens reviewed itself: on !18, 1,033 changes, every one of the 32,881 changed lines shown, checked before and after the agent's note.
The release gate held, even after the revert, until it learned to net a change against its revert.
The not-the-author rule held on a real merge request, !13.
The incident loop closed on real runs: a warning, a bypass, a failed watch, Incident #3, the revert, merged by a person.

What we learned

The agent is most useful when code has already decided the structure: what changed, what connects, and whether every line is shown. Then the agent only has to explain, and CI can check it. Developers also want the tool they already know: a familiar diff with the links on the code beat every new kind of picture we tried.

What's next for Molly

Show a person clearing a history risk live, and turn on the review-lens flow's Mention trigger so reviewers can ask questions from the page.
Run on Snowflake or BigQuery, with base and head as two databases per merge request.
Learn which kinds of changes each data owner approves, so routine ones can be pre-approved.