Paciacore

Inspiration

Climate commitments rarely die in a headline retraction. They die quietly — a "for" dropped from a sourcing pledge, a hard percentage swapped for a softer phrase, a scope boundary that narrows one filing at a time. Read any single year in isolation and it looks like progress. Line up three years side by side and the pattern is unmistakable: the language gets more confident while the substance gets smaller. Nobody catches that by hand, because nobody cross-references three years of filings unless it's literally their job.

The tools that already exist don't close that gap. The groups doing this kind of watchdogging by hand — InfluenceMap, DeSmog, SourceWatch, the analyst teams at CDP — catch real mutations, but manual doesn't scale and isn't easy to audit afterward. The raters that do scale — MSCI, Sustainalytics — score a disclosure at a single point in time, so a company can soften its language every year and never lose a point, because the scoring was never built to notice softening. Nobody sits in the middle: automated, longitudinal, and able to show its work. The timing makes it sharper, not softer: Canada's own Competition Act now has real anti-greenwashing teeth — Bill C-59 added a reverse-onus substantiation requirement in 2024, and since June 2025 any individual or group can bring a greenwashing complaint directly to the Competition Tribunal, with penalties up to $10 million or 3% of global revenue. Bill C-15 already loosened part of that standard in March 2026. Two amendments to the same law in two years isn't a settled bar, it's a moving one — and a moving legal bar is exactly the kind of thing a system built to track how language changes over time should be watching. That's why we built PACIA (Provenance Aware Compliance And Inconsistency Analytics).

What it does

PACIA reads years of an institution's filings and tracks how its claims change — not just what a report says, but how the language moved since the last one. It's really two systems. The first is a pipeline: a document goes in, claims get extracted, and each claim is argued over by five specialist agents — reading for narrative framing, drift, historical context, evidence, and logical coherence — through a debate that almost only moves forward (one state, a strong defense, can still draw renewed challenge) and that a judge closes with one of four verdicts: accepted, rejected, contradicted, or genuinely unresolved. When a verdict is headed toward accusing an institution of something serious, a sixth agent checks whether an earlier document already disclosed the same fact before the claim gets to publish.

The second is a collaboration: an agent, running on Gemini 3.7 Flash through Google's Gen AI SDK, that works turns with a user — pulling claims from the pipeline, checking them against what it already knows, and resolving the spatial and geographic details behind a claim (grounded via Gemini with Google Maps) before it synthesizes an answer — on its own initiative within a turn, not step-by-step on command. One of its capabilities is getting better at that job over time: it watches its own performance and, through a separate, tightly governed process, can update how it operates when a change is genuinely justified — never on a whim. The result, live at paciacore.com/feed, has already caught real drift: Nestlé quietly dropping the binding word "for" out of a sourcing commitment, Nepal swapping a measurable 15% target for the vaguer "triple renewable capacity."

How we built it

On the pipeline side: a document is chunked and reduced into a document-specific schema so long filings don't get truncated, and every extracted claim leaves carrying its own provenance seed — which document, which sentence, which step produced it. From there it goes to five specialist agents at once, arguing it out through states that almost only move forward — proposed → challenged → defended → refined, with one deliberate exception letting a strong defense draw renewed challenge — closed by a judge that scores evidence quality, consistency, provenance, and confidence. We added the sixth, budget-limited audit agent after a real miss: the live feed flagged LNG Canada for a "dropped commitment" on wetland impact, externally validated, off a real difference between LNG Canada's two most recent compliance reports — and it turned out LNG Canada's own public Wetland Compensation Plan, filed under that same permit number years earlier and revised through 2020, had already disclosed the same fact. The five agents just never had that third document.

On the collaboration side: this isn't a system that just sits and watches. The governance loop follows Google's Agent Development Kit patterns — reimplemented against our own ports and adapters rather than imported as a dependency. It runs its own turns — pulling in claims from the pipeline, checking them against what it already knows, resolving the spatial and geographic details behind a claim where that matters (via Gemini grounded with Google Maps), and synthesizing an answer — autonomously within a turn, without being walked through each step. Getting better at that job over time is a capability layered on top of it, not the point of it. A metacognitive layer watches its own performance quietly, on purpose: before it's trusted to actually change how any agent behaves, we wanted to see how it behaves itself, so for now it can only propose — it can't write anything back, not partially, not under any condition. Every proposal it does make has to clear five independent statistical checks, run by a completely separate governance process, before it goes anywhere near a live agent: does it help, is that improvement statistically real across repeated runs, is it reproducible, does it avoid breaking anything that used to work, and does it still cite evidence as often as before. We tested that boundary end to end on Gemini 3.5 Flash-Lite and a local Ollama model proposing an update to the same skill. One passed all five checks — genuinely better, statistically solid — and still didn't ship, because deployment runs through a separate simulator step that blocks it by default. The other looked great on every easy-to-read metric while the one number that actually mattered, whether it got the right answer, fell by more than half — it had learned that saying "nothing to see here" made its dashboard look fantastic while quietly abandoning the job. That's also why the pipeline's own judge scores confidence as its own dimension. It's also why we're leaving agent reputation neutral for now rather than letting it drift on an unverified track record: reputation-weighted trust is only honest once there's a real feedback loop confirming outcomes, and that loop isn't wired in yet — so scores stay at neutral until it is, rather than start meaning something before we can back it up. Underneath all of it sit a few smaller checks that make the whole thing trustworthy enough to call self-evolving: one confirms a proposal's cited evidence is real and not invented, one lets the agent fetch its context faster without changing what it finds, one replayed dozens of past cases to confirm nothing gets written outside governance's say-so, and one throws out any suggestion whose underlying document changed while it was still in flight. None of these pieces does much on its own. Together, they're what let this be a collaborative partner that keeps learning on its own initiative and only ever updates a skill when the case for it is genuinely strong — not because it felt like it.

Challenges we ran into

The hardest problem we didn't see coming: a debate can be completely correct about a textual difference between two documents and still be wrong about what that difference means, if a third, earlier document exists outside the two it was given. That's not a parsing bug, it's a scope bug — and there's no way to "just ingest everything," because there's no upper bound on how much an institution has ever published. The fix had to be narrower: verify the specific identifiers a claim already cites, not chase full document history.

The other real challenge was building a governance process we could actually trust. It's tempting to grade a proposed skill update on its easy-to-read metrics — did it cite more evidence, did it interrupt people less — and one of our two candidate updates looked fantastic on exactly those numbers while its core accuracy collapsed by more than half. Catching that meant refusing to let any single metric, including statistical significance itself, be the whole judgment. And underneath both of these: five agents per claim, multiple debate rounds, across a corpus running thousands of pages, gets expensive fast if you're not careful — we ended up caching judge instructions, trimming later debate rounds down to just what changed, and letting high-confidence claims resolve in round one without triggering the round-two re-dispatch that contested claims get.

Matching a claim across two filings turned out to be harder than it sounds — less a single needle in a haystack than the same needle hidden in two separate haystacks, and a lot has to go right in the path from raw PDF to a matched pair before the answer is trustworthy. Code being theoretically correct doesn't mean it behaves correctly once real filings hit it. That difficulty compounds with a subtler methodological problem: knowledge-cutoff decontamination. Well-known cases like Nestlé and China Three Gorges are exactly the kind of data a large frontier model may have encountered during pre-training. If we validated the system on such a model, any "catch" risks being recall from training data rather than genuine detection. So we deliberately validated on older, cutoff-appropriate models whose training data predates our corpus, and checked every result against ground truth — ensuring that a caught claim was actually caught by the system's reasoning, not remembered by the model's weights.

There is also a gap between what we can demonstrate at a hackathon and what the system actually does in practice. The system's real strength is speed: within hours of a new ESG report being published, it can surface claims that could be troublesome — contradictions, quiet target revisions, commitment regressions — before anyone else has flagged them. But that very capability is what makes a live hackathon demo difficult. The most compelling catches are from reports so recent that no third party has independently validated them yet, and making a first-ever public claim about a real company's ESG drift at a hackathon would be inappropriate. So the demo uses older, already-corroborated cases where ground truth exists, even though those are not the frontier the system is built for. The collaboration layer faces a similar constraint: we can't ingest every document that exists, so the system is designed around an analyst in the loop — the collaboration helps the analyst, it doesn't replace them. We ran simulations for the demo, but genuinely evolving the collaboration layer takes real usage with real analysts over real time; you can't fast-forward that by replaying historical data. Early results are promising, but the layer needs significantly more testing before we'd trust it on its own.

Accomplishments that we're proud of

The system sees mutations that are genuinely hard to catch by hand — Nestlé quietly dropping a word, Nepal swapping a target number, China Three Gorges downgrading flood-control scope behind identical headline language. Those are historical cases we validated against ground truth. LNG Canada is not historical — it's current, the reports are actively being scrutinized by others right now, and we're pulling a single matched claim out of filings that run into the hundreds of pages each and flagging the drift correctly. That's a live event, not a backtest.

On the governance side: the full EVE lifecycle ran end-to-end on two very different models and correctly kept both out of production for two completely different reasons — one rejected outright by the gates, the other validated on paper but stopped by the simulator. That's the multi-gate design working as intended. Underneath it all, over a hundred passing tests across the orchestration modules — not a side artifact, but the reason we trust what the live findings are telling us.

What we learned

Two design decisions did a lot of quiet work. Hexagonal architecture — ports and adapters — was the biggest one: it kept integration simple for the coding agent, since it only ever has to reason about a port and an adapter rather than the whole system, and it let us swap or discard entire components without disturbing the rest. That wasn't something the coding agent arrived at on its own; it was a decision we made going in, and it's a large part of why the build moved as fast as it did. The microkernel behind PACIA's orchestration — the blackboard architecture — was a smaller, similarly deliberate choice, picked explicitly to keep the system disciplined rather than left for the agent to improvise.

We also came away wary of confidence laundering — the risk that agents talking to each other, round after round, quietly inflate each other's confidence or let provenance drift without anyone deciding that should happen. We leaned on a scatter-gather pattern to guard against it, and treated provenance as something that has to be actively preserved at every handoff, not something that survives agent-to-agent conversation by default. We still want to check further whether any laundering is slipping through undetected.

Built during this hackathon

Not everything here started from zero this month. The idea that claim extraction needed provenance baked in from the start, not bolted on after — the conceptual core of the pipeline — came out of earlier brainstorming and some early experimentation on the extraction side before the hackathon began; we didn't arrive at "provenance-aware" as an afterthought, we knew vanilla RAG was not going to work for us (We don't know what we are looking for only that some claim mutates and people mutate them very cleverly). So we had some ideas of what we were doing but we "created" in the hackathon period. The collaborative partner, was conceptualized and built entirely during the hackathon period.

What's next for Paciacore

This was a one-month hackathon, and there's a lot more validation we want behind it before we'd call it done. The governed evaluation alone already ran 340 calls per model — 680 total — and claim extraction has been validated against ground truth across four companies' filings, including one live test run entirely on a knowledge-cutoff model that had never seen that company's ground truth in training — a model that had seen the samples during training could just be recalling them rather than actually detecting anything, so the older cutoff is what makes that test mean something. But the collaborative partner piece was built in about two weeks of that window, and two weeks isn't enough to be confident in something meant to sit next to an actual analyst's judgment — so more testing there, specifically, is the priority before anything else.

We don't think the next step is more features. The architecture's already built to take them — that's what the hexagonal design is for — but adding capability isn't what's missing right now; confidence is. We're in the process of incorporating the company, and the most useful thing we can do before then is get it in front of many ESG analysts: we're already sharing demo samples with a few and asking directly what would actually make this useful to their work, rather than guessing at it ourselves.

Built With

• antigravity
• claude
• codex