-
-
Control Room, bound to PAY/ITO: 18 verified, 1 stale, 1 missing, 0 not visible, 0 not checked (20 claims), 17 graph reads.
-
PAY-1 Evidence Gate: evidence complete - all 4 claims backed by evidence the graph could see, plus the guarantees boundary.
-
PAY-1 Approval accountability, verified (member of Team Pulse): four-hop Teamwork Graph trace, query key, rows, ms.
-
PAY-6 Gaps found: Incident Runbooks last edited 1,936 days ago against a 180-day window -> stale, owner not resolved.
-
Audit pack: fingerprint -> VERIFIED MATCH, Unexplained gaps: 1 of 2, AC-06 row carrying its exception until 2027-01-31.
Inspiration
I run infrastructure programmes for a regulated payments business. Every quarter someone from compliance or an external auditor asks a variation of one question: "prove that this control actually operated last quarter." Not "is it configured", not "does the policy exist", did the twelve production changes we shipped in October each carry an approval by somebody other than the person who shipped it, under a policy that is still current, owned by a team that still exists.
The evidence for all of that already exists. It is in Jira tickets, in approval comments, in the Confluence pages those tickets link to, in the space owners and the teams behind them. What does not exist is anything that puts those together. So the work is done by hand, in a spreadsheet, every quarter, under deadline, and it is done by a person who is not an auditor, reading tickets at speed. And it hides a second, worse problem. When you cannot read something, because you lack a permission, or a page sits in a space you cannot see, a spreadsheet has no way to say so. The cell goes empty and the row reads "no evidence". In a regulated institution, "we could not see it", "we could not check it this time" and "it is not there" are three different findings. Only the last one is the audit problem, and only the last one should ever be filed as one.
The Teamwork Graph API opened in preview on 15 October 2026, twelve days before this hackathon started. It is the first time an app can traverse work item -> policy page -> owner -> team across Jira and Confluence in one query and say who answers for this object, across products. That is the capability this problem needed all along.
What it does
AssureLoop is a Forge app for Jira with four surfaces:
1. The Evidence Gate (Jira issue panel): the product. Open any work item and the app judges, live, every claim its control requires: the completed change record, an independent approval comment, a governing policy page inside its freshness window, a test or attestation record, a linked pull request, a deployment record. No scan step stands between you and a verdict. Each claim gets one of five states and a reason a person can act on. 2. The Control Room (Jira global page). The rollup: the last scan's counts, the delta against the previous scan, scan integrity (retries, claims not checked, claims out of view), an Environment Check, the audit pack, the gap list with exceptions, the reverse scan, and the control registry. It is also where a judge binds the shipped programme to their own project and space in one click.
3. The audit pack (Confluence macro). A workpaper, not a dashboard: one row per claim with the evidence link, the observation date, its window, the remediation issue, the state and the reason. The header says as of <scan finish time>; the footer carries a SHA-256 fingerprint over all 18 fields the pack hashes. Verify fingerprint rebuilds the pack from the same scan, re-hashes it with the same function, and reports VERIFIED MATCH or DRIFT DETECTED. Copy the audit memo puts the deterministic text (the same rows always produce the same words) on the clipboard.
4. The Assurance Agent (Rovo). Four actions (get-evidence, explain-verdict, scan-control, create-remediation) and one reusable skill (audit-pack-playbook). The agent never decides a state: it reads the engine's verdict, narrates it with the graph path, and performs the write a human confirms.
The loop actually closes: each gap becomes one Jira issue, assigned through the owner route the app resolved (graph page owner -> issue assignee -> component lead -> control owner), with that route written into the issue description so the next reader knows how the assignee was chosen. Re-scanning does not open a second issue for the same claim: each remediation carries a finding token in its summary ([AL-xxxxxxxx]), and the token is matched before anything is created. A work item that was only read as a template (no control label) never writes.
And the honesty is a first-class feature, not a disclaimer:
| State | Meaning |
|---|---|
VERIFIED |
Evidence exists, is fresh, and its owner is active |
STALE |
Evidence exists but is past the claim's window, or its owner is deactivated |
MISSING |
Evidence really is absent, and the scan could see the data it looked at |
NOT_VISIBLE |
A Teamwork Graph scope was not granted; the row names the scope to approve |
INCOMPLETE |
The scan could not check it this time: throttle, timeout, index lag, teamsV2 403 |
How we built it
Platform and data. One Forge app (nodejs24.x, sql, storage:app) with 13 scopes, no external egress, no container, no LLM call in the verdict path. Forge SQL holds the control registry (12 controls, 20 claims shipped with the app), evidence links, scan history, findings, provenance, remediations, exceptions and app config, nine idempotent tables, one migration runner. 20 resolver keys serve the three Custom UI surfaces and the actions.
Teamwork Graph. Curated queries where the catalogue covers the hop (jiraSpaceWorkItems, jiraWorkItemComments, confluencePageOwners, atlassianTeamsBySpace, atlassianUserInTeams), and one persisted Cypher query (assureloop.evidence-chain, registered against this app and executed by queryKey) for the work item -> policy page -> space -> team chain the catalogue cannot express. A persisted result is treated as optional enrichment: it is recorded as a note on the trace, and it can never move a verdict. Without a queryKey the app runs curated-only and says so.
Jira REST enumerates, the graph relates. JQL is how the population of work items a control is tested against is listed precisely; the graph is how work item -> page -> space -> team is proven. Every row shows which of the two answered.
Failure is never a verdict. runGraphQL retries throttles and timeouts with exponential backoff, honouring Retry-After; when retries are exhausted the failure travels with the observation and the claim becomes INCOMPLETE. MISSING requires a read that finished and saw nothing. A scope is PROVEN only when a query that reads it returned rows. An empty result is INCONCLUSIVE, because the API fails closed. That single rule is what stops a permission gap from being filed as an audit finding.
The unattended scan. A scheduledTrigger at interval: day with a "at least 20 hours apart" guard (ASSUR_SCHEDULED_MIN_HOURS, default 20). A scheduled run has no user, and asApp() cannot call the graph, so every hop is given the stored scan principal as its identity, a 6-second cap, and a circuit breaker after two consecutive caps. A capped hop becomes TIMEOUT (never MISSING) and the scan falls back to Confluence's own owner fields, labelled rest, before it gives up. The scan record keeps one line, graph 17 read(s), 17 as the scan principal, identity warm-up 361 ms, so the Control Room can show how the unattended run read the graph.
The Environment Check turns the app's blind spots into the first screen: six probes in a single 5-second budget (connection, work item, comment, page/space, user/team, persisted query), each green / yellow / red, each carrying the scope to approve or the action to take.
Exceptions and the reverse scan. A gap can carry a reason, an expiry and a compensating control; recording one never changes the verdict: it moves the gap from unexplained to explained and discloses it on the pack with its author and expiry. After every scan the app also reads the programme's spaces and subtracts every page a control names. What is left is evidence nobody asserts, which it reports as rest+registry, saying so rather than implying a graph traversal it does not make.
Empty sites. The demo's PAY project and ITO space are not on a judge's site, so the Control Room offers Bind the programme: pick a project and a space, press once. The twelve controls' project key is rewritten in Forge SQL, every policy rule that names a space this site does not have is repointed to the chosen space (a rule that already names a space here is left alone), and the demo's evidence links are retired: a link to a page this site does not have is not evidence. Pages the chosen space does have are re-confirmed by title and re-linked, so the pack keeps their names. No new scope is needed. A project with no control labels is judged in discovery mode: the five most recently completed changes, and every row says so. A missing project is reported as a setup step (UNCONFIGURED), not as a permission gap.
Testing. npm run smoke runs 489 assertions with no site and no credentials: verdicts, failure classification, a gate judged against a fake Atlassian, scan honesty, the fingerprint's 18-field coverage and its drift regressions, SVG trace layout in jsdom (no overlapping boxes, no clipped labels, backward hops get their own column), the Control Room's scan button actually calling the resolver, resolver-key uniqueness, and the graph budget (a hung hop returns instead of blocking; a capped hop is not retried). TypeScript is strict and clean; forge lint reports no issues.
Where to look next. The source is public at https://github.com/luke-shin-ai/assureloop-for-jira, and a static, no-install version of the Control Room runs at https://luke-shin-ai.github.io/assureloop-for-jira/demo/. The static page is the recorded output of the 2026-11-20 scan rendered by the same front end, so a reader can open every claim row without installing anything; the app itself is what runs a scan against a live site.
Challenges we ran into
A missing scope does not error. It returns an empty, successful result. This is documented behaviour and it is the single most dangerous property of the graph for a compliance product: the natural implementation reports "no evidence" everywhere and looks like it works. We had to build a scope ledger whose only proof is "this query returned rows", and then make an unprovable claim NOT_VISIBLE rather than MISSING.
Our own unattended scan lied for one day. The scheduled run called the ownership hop without a user context, got failures, and reported six controls as STALE ("nobody owns this evidence"), which is a finding, not a read error. We fixed the class of bug, not the instance: an unread hop is now INCOMPLETE, and every hop in an unattended run reads as the stored principal with a hard cap. Turning that fix into a measurement (17 read(s), 17 as the scan principal, 0 capped) took a temporary fiveMinute trigger, because waiting a day per experiment is not a debugging loop.
Trusting a scope change to demonstrate itself. Our original plan for NOT_VISIBLE was to revoke a scope on camera and rescan. It does not reproduce: changes take about a minute to propagate and rows keep arriving after removal. The honest versions are a deliberate partial install, a site where the admin withheld the scope, or a genuine 403, and the video now says which one it is showing.
Two databases, one product. Forge SQL is scoped to the installation that invokes it, so the Jira installation and the Confluence installation each keep their own registry and scans. A judge comparing the macro and the Control Room will see different unexplained-gap counts. Rather than hide it, the documentation explains it and the macro has its own Scan now (this installation).
Ownership data that is not there. Confluence page ownership returns zero rows on our demo site, and teamsV2 can answer 403 while everything else is granted. The app now tries the graph, then Confluence's own owner fields (ownerId -> lastOwnerId -> authorId, provenance labelled rest), and if all of them are empty it prints owner not resolved instead of implying an owner exists.
Platform cold starts, and a dead button. The first invoke after idle can take 70-180 seconds; one round of testing had Verify fingerprint return Failed to fetch twice in a row. Separately, the Control Room's scan button did nothing at all for a while: a local const scan = rollup.lastScan had shadowed the click handler. Both are now regressions in the suite: the page pings on load and invokes retry once on a transport failure, and a jsdom test asserts that clicking Scan the registry really calls scanNow.
Proving a table was not edited. A SHA-256 over a few visible columns proves almost nothing. The fingerprint now covers all 18 fields the pack hashes (evidence label, URL, missing scopes, exception reason, expiry, compensating control, remediation, judged time, age and window), with row order and clock-derived values excluded, and 18 regression assertions that edit nine of those fields and prove both that the pack reads DRIFT and that the hash value moves.
Accomplishments that we're proud of
- A five-state verdict model, enforced everywhere: panel, registry scan, agent, audit pack, with
NOT_VISIBLEandINCOMPLETEas first-class states. In a regulated institution that is not a UX nicety; it is the difference between a finding and a fabrication. - Provenance you can read: every claim expands to issue -> comment -> approver -> policy page -> team, each hop with its ARI, relation, observation time, and a footer with the query key, row count, latency and route that answered it.
- A workpaper an auditor can check: 20 claim rows, a fingerprint over 18 fields, one-click VERIFIED MATCH, one-click deterministic memo, and a trust boundary printed on every surface: what the pack guarantees and what it explicitly does not.
- A loop that closes: 1 stale and 1 missing claim on the demo programme, two remediation issues created, each with its owner route in the issue description and a finding token in its summary, and
Unexplained gaps: 1 of 2once the stale one carries a recorded exception. - An unattended scan that is honest about itself: 19,587 ms, 17 graph reads as the scan principal, 0 capped, 0 retried, and the same verdicts as that day's manual scan.
- 489 assertions that pin the claims above, including the ones that were bugs first.
Runs on Atlassian eligible with no egress: no data leaves Atlassian, nothing to procure, nothing to bill.
What we learned
The interesting engineering in compliance tools is deciding what not to claim. Every state we added (
NOT_VISIBLE, thenINCOMPLETE) came from a case where the honest answer was "I do not know", and each one removed a way the product could tell a confident lie.Deterministic verdicts plus a model that only narrates is a much better fit for regulated users than the reverse. The agent is genuinely useful (routing, explaining, drafting the write), but no model output can change a state, and that is checkable in the code and in the tests.
A quality bar needs a fixture, not a demo site. Half of the bugs above were only reproducible against a fake Atlassian with a scripted 403, a scripted 429 and a 412-day-old page.
Unattended runs are a different product from interactive ones. No user context, no warm caches, timeouts you cannot see. It has to be designed for, capped, and made visible on screen.
Documentation drift is a real defect. Several review rounds found the README promising modules and flows the code did not have. A judge who opens the manifest after the video will find every one of them.
What's next
- Snapshot a pack to a Confluence page so a filed workpaper can be frozen rather than re-verified every time (needs
write:page:confluence, hence a major version and a re-approval, planned as the release after the submission period). - Move schema migration to an
avi:ecosystem:installed:appconsumer, so the first user request of a fresh install is not also a migration. - Drop the privileges we do not use: the unused impersonation grant on
write:jira-workand the legacyread:confluence-content.allscope. Both are held today only to avoid a mid-submission major version that would force every reviewer's installation to re-approve permissions. - More evidence types where connectors exist: pull requests, deployments and attestations are already first-class evidence types in the engine; each becomes a hop when the site has the data.
- A non-blocking workflow hint for the ship gate ("this change has an unverified claim"), which needs no validator and no admin configuration.
- Programme import so a control library can be loaded from a spreadsheet instead of by hand. The thing this app replaces should be the last thing it asks for.
Built With
- confluence
- cypher
- forge
- graphql
- jira
- node.js
- rovo
- teamwork-graph
- typescript