Forkward

Every upstream fix, forwarded to your fork. AI agents propose conflict resolutions; you review them in a pull request.

Comment

Inspiration

A fix upstream is not yet a fix in your fork.

Companies carry long-lived forks to support their own hardware, products, and customers. Pulling upstream changes into those forks can mean resolving the same kinds of merge conflicts over and over. The longer a team postpones the work, the longer security and stability fixes wait to reach its users.

We built Forkward to turn that maintenance into a repeatable workflow: propose a resolution, show the evidence, and let an engineer review it. The hard part is deciding when an automated answer deserves trust.

What it does

Forkward merges upstream into a new branch of your fork, proposes a fix for each conflict, and opens a pull request. It works through a web app, a CLI, and an MCP server that coding assistants can call.

For each conflicting hunk:

  1. Seven deterministic rules propose candidates without a model call. They can keep the fork, take upstream, concatenate the sides in either order, replay one side's change onto the other, or delete the block.
  2. Three model personas propose independently. The frozen strategy uses the same model with fork-first, upstream-fix-first, and refactor-aware briefs. Identical answers pool their votes. A separate classifier predicts whether to keep the fork, take upstream, or combine changes.
  3. A safety gate checks the evidence. Automatic application requires a clear vote, agreement from an eligible deterministic operator, classifier agreement, and a confidence threshold calibrated for that provider and model.
  4. A person reviews the rest. The web app shows both sides, the proposal, alternatives, and why the gate deferred. Engineers can accept, edit, or reject resolutions. Repository CI checks the resulting branch.

Auto-apply is disabled today. The gate has not met its certification target, so every conflict requires human review. Forkward proposes changes on a separate branch; it does not merge them into the default branch for you.

Merged resolutions feed a cache so a recurring conflict can reuse the approved answer. During training, a reflector and curator turn failures into a versioned playbook. Strategy search keeps improvements only when validation gains exceed the estimated noise.

How we built it

The engine is Python 3.11+ with uv. FastMCP exposes resolve_conflicts, open_sync_pr, get_job, and get_updates; DBOS runs durable workflows on Postgres. Tenant tables use forced row-level security.

The frontend uses Next.js 16, React 19, and Better Auth for GitHub sign-in and MCP OAuth. A GitHub App creates sync pull requests and posts gate checks. The live job view connects each conflict to its proposals and review decision.

We implemented Gemini and Vultr inference backends alongside Luminal, DeepSeek, Anthropic, and local OpenAI-compatible servers. Auth0 can authorize MCP clients. A TimescaleDB migration supports time-partitioned run logs and hourly aggregates when the extension is available. Deployment uses Caddy and systemd on a Linux VM. Stripe metering, OpenTelemetry, and Langfuse support billing and observability.

Optional Solana devnet anchors record a digest of sync decisions when the PR opens, then a linked digest of the review outcome when it merges or closes. The code can recompute the digests to check them against the memos. This makes changes to the recorded evidence detectable; it does not prove that a resolution is correct. Optional ElevenLabs support produces spoken scheduled-sync digests.

Challenges we ran into

Agreement is useful evidence, but it is not a correctness guarantee. Several personas can make the same mistake, and a deterministic rule can be wrong for a particular conflict. We separated proposal generation from the gate's permission to apply a change, and made failed calibration produce abstention.

A second challenge was evaluating improvements honestly. We trained on 70 NXP hunks, validated on 45 STMicroelectronics hunks, froze the strategy, and evaluated on 71 Rockchip hunks held out from tuning. Keeping the fork's code is a strong baseline on this test set; a large improvement over zero-shot is only part of the picture.

Turning the resolver into a service also required durable jobs, tenant isolation, GitHub webhook handling, and keeping each human review tied to the branch commit it changes.

Accomplishments that we're proud of

We built a working public conflict-review demo backed by a recorded run. Its four conflicts were resolved using DeepSeek V4.1 Flash through Vultr inference. Three passed the other gate checks, but all four still required review because the model's gate was not certified. The demo exposes every proposal instead of showing only the selected answer.

The saved Rockchip benchmark reports:

Approach Matches Accuracy
One zero-shot model call 33 / 71 46.5%
Always keep the fork's side 51 / 71 71.8%
Forkward's frozen strategy 54 / 71 76.1%

These benchmark results used DeepSeek V4.1 Flash through Luminal, separate from the Vultr demo. Matching uses the repository scorer's normalization: it strips surrounding whitespace on each line and drops blank lines. It measures agreement with engineers' recorded resolutions, not semantic equivalence or security.

Forkward gains 29.6 percentage points over zero-shot, but only three hunks over keeping the fork. That smaller edge is not statistically significant: the exact paired McNemar test gives p = 0.629. The saved token totals are approximately 4.16 times the zero-shot baseline, so the accuracy gain also has a cost.

An eight-hunk timing sample recorded 4.71 seconds per hunk sequentially versus 1.89 seconds in parallel, a 2.49× speedup. This is a small timing measurement, not an end-to-end production latency guarantee.

What we learned

The strongest lesson was that a useful resolver and a certified automatic gate are separate milestones. The historical experimental gate accepted 18 test hunks and got 8 of those wrong. We do not present those acceptances as safe automation.

Even with zero errors, a fixed threshold would need at least 59 accepted calibration hunks to support a one-sided 5% error bound at 95% confidence. Our entire validation set contains 45 hunks. Current product logic refuses thresholds whose calibration target was not met, and the saved Vultr calibration accepts zero hunks.

We also learned to compare against simple rules, keep provider-specific calibration separate, and make uncertainty visible in the review interface.

What's next for Forkward

Expand the independent evaluation and calibration data, compare against compute-matched baselines, and measure the full sync-to-review workflow across more kinds of forks. Keep auto-apply disabled until the relevant provider/model calibration meets its target.

We want approved resolutions to reduce repeat maintenance work, while linked decision and outcome records make the gate's evidence easier to inspect over time.

Try it

Open the recorded demo without signing in. Select Makefile, then include/ringbuf.h, to compare a proposal without deterministic confirmation against one that passes the other checks but still waits for certification. Inspect the alternatives and the recorded timeline. The preview does not create a PR or modify a repository.

Built With

Share this project:

Updates

Submission history