PRemory

Inspiration

I keep abstracting lines of code that are only used once when I should be inlining them. Reviewers repeatedly point this out, but that feedback disappears after a pull request is merged. We realized the same problem exists across entire engineering teams: valuable conventions are buried in years of review comments and repeatedly taught one pull request at a time.

What it does

Integity transforms historical pull request reviews into structured engineering memory.

It ingests merged PR comments, connects rejected code with accepted fixes, and extracts reusable repository-specific conventions. AI coding agents can query this memory through MCP before writing code and validate their proposed diff afterward.

Integity also provides pre-commit and VS Code feedback, helping developers catch likely review blockers before opening a pull request. A dashboard shows recurring feedback, convention frequency, accepted-fix coverage, and reviewer activity.

How we built it

We built Integity as a TypeScript monorepo with several connected components:

A GitHub ingestion pipeline collects review comments and code context from merged pull requests.
An extraction engine links comments, rejected snippets, and accepted fixes into review episodes.
A Freesolo-trained Qwen3.5-4B adapter classifies review intent and produces structured detection metadata.
Deterministic grounding validates every generated code signal against historical evidence.
PostgreSQL and Prisma persist conventions, evidence, confidence, scope, and detection rules.
An MCP server exposes repository conventions and predicts feedback on proposed diffs.
A GraphQL API connects the memory engine to the VS Code extension and dashboard.
A Next.js dashboard provides a simple, Vercel-deployable view of review analytics.

The retrieval and validation system combines repository, path, and language filtering with confidence, support count, exact code signals, and semantic fallback.

Challenges we ran into

The hardest problem was preventing false positives. A review convention may only apply inside a particular file, function, or architectural layer. Searching for a prohibited word without context would create noisy and untrustworthy feedback.

We addressed this by separating:

Trigger signals that establish context
Forbidden signals that represent rejected code
Required signals that must appear in that context
Semantic-only rules that cannot be safely reduced to substring matching

We also found that the trained model could return semantically useful answers with inconsistent JSON. We simplified its output contract, balanced the training data, added grounding and behavioral replay, and automatically downgrade unsafe detections to semantic-only conventions.

Connecting historical review comments to the actual merged fix was another challenge. A nearby code change is not necessarily proof that the reviewer’s request was implemented, so linkage quality must remain explicit.

Accomplishments that we're proud of

We built an end-to-end system that turns real review history into infrastructure that coding agents can use.

We are especially proud that:

MCP returns compact conventions rather than dumping raw GitHub data into an LLM.
Every executable signal must be grounded in supplied code.
Generated detections are replayed against rejected and accepted examples.
Unsafe results degrade safely instead of becoming false-positive blockers.
The system continues working through deterministic fallback when the hosted model fails.
Developers can receive feedback through their existing editor and commit workflow.
The dashboard makes previously invisible engineering patterns understandable to a team.

What we learned

We learned that engineering memory is not the same as retrieval. Finding a similar review comment is useful, but transforming it into a reusable rule requires evidence, context, confidence, and a known detection strategy.

We also learned that model quality cannot be measured by training loss or exact JSON matching alone. The important test is behavioral: does a convention catch the historically rejected code while allowing the accepted fix?

Finally, we learned that AI works best here as one component of a larger system. The model interprets review intent, while deterministic code handles evidence grounding, persistence, scoping, validation, and safety.

What's next for integity

Next, we want to:

Ingest more repositories and higher-quality accepted-fix links.
Train on hundreds of human-reviewed examples across languages and frameworks.
Add AST and language-server context for more precise detection.
Learn team-, repository-, directory-, and developer-specific conventions.
Improve convention clustering and identify rules supported by multiple reviewers.
Add GitHub App support for automatic merged-PR ingestion.
Measure whether Integity reduces repeated review comments over time.
Let teams approve, edit, suppress, or promote learned conventions.
Integrate with more coding agents and CI systems.

Our goal is for Integity to become a durable memory layer that helps both developers and AI agents learn how a codebase is actually built.