-
-
User flow: five steps from confirming the plan to release, and what closes release again
-
Block diagram: plan and attached files go into one rules engine; every surface reads its verdict
-
Architecture: Forge modules, functions, shared source modules and the Atlassian platform
-
Settings: admins choose who may approve and can require a second person to sign off.
-
Describing a change: results are read from the attached results file, with the plan's limit beside each one.
-
Confluence: Envelope proposes structured rules from the plan page. A person checks every value before confirming.
-
Register: every change and a hash-linked audit trail, checked on each load ("Chain intact"), exportable as CSV.
-
Jira: the confirmed plan on the work item, with each planned modification, its limits and required evidence.
-
An evidence file was removed after sign-off: verdict drops to evidence missing and release is refused.
-
Release refused: the change is inside the plan, but nobody has signed it off.
-
Rovo agent answering "Is this work item ready for release?" from the same rules.
-
The plan page was edited after confirmation: release closes for every change until the plan is confirmed again.
-
Verdict: inside plan. Each protocol check shows what was required, what was found and which file it came from.
-
Fresh site, team-managed space: the release gate is offered as a workflow rule
-
Fresh site: moving an undescribed change to Done is refused by the gate
Inspiration
A team that ships an AI model inside a medical device cannot simply retrain it. A change that could affect safety or effectiveness normally needs a new marketing submission. Since December 2022, US law offers another route: agree a Predetermined Change Control Plan with the FDA in advance, and a change made according to that plan needs no new submission [1]. The FDA's guidance for AI-enabled devices sets out what the plan must contain: the planned modifications, the protocol for developing and validating them, and an assessment of their impact [2]. The FDA, Health Canada and the UK's MHRA jointly published five guiding principles for these plans; the first is that a plan is "focused and bounded" [3].
Three facts made this worth building:
- The audience is large. The FDA lists over 1,600 authorized AI-enabled medical devices as of September 2026 [4].
- The mechanism is underused. Of 794 AI-enabled devices authorized from 2023 to 2025, only 43 (5.4%) had an authorized plan. The authors conclude that there "remain unidentified barriers" to using them [5].
- It is not only an American or medical idea. Article 43(4) of the EU AI Act says that changes to a high-risk AI system "pre-determined by the provider" at the initial conformity assessment "shall not constitute a substantial modification" [6].
A plan only helps if every later change provably stays inside it. Today the plan is a document in Confluence and the work is in Jira. Nothing connects them. "Is this change inside the plan?" is answered by a person reading a PDF, once per change, and the proof is a spreadsheet. We wanted the plan to be a rule the workflow enforces.
Sources
- 21 U.S.C. § 360e-4 (Food and Drug Omnibus Reform Act of 2022): https://www.law.cornell.edu/uscode/text/21/360e-4
- FDA final guidance, "Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions": https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence
- FDA, Health Canada, MHRA, "Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles" (2023): https://www.fda.gov/medical-devices/software-medical-device-samd/predetermined-change-control-plans-machine-learning-enabled-medical-devices-guiding-principles
- FDA, Artificial Intelligence-Enabled Medical Devices (current as of 22 September 2026): https://www.fda.gov/medical-devices/digital-health-center-excellence/artificial-intelligence-enabled-medical-devices
- RAPS, 29 May 2026, on Dabek et al., JAMA Health Forum: https://www.raps.org/resource/study-finds-low-pccp-utilization-among-ai-enabled-devices.html
- EU AI Act, Article 43(4): https://artificialintelligenceact.eu/article/43/
What it does
Envelope is one Forge app installed across Jira and Confluence. Its users are the regulatory or quality lead who owns the plan, the ML engineers who propose changes, and the approvers who sign them off.
- Reads the plan where it lives. On the plan's Confluence page, "Use as change control plan" proposes structured rules from the text: each planned modification, its permitted data sources and populations, its acceptance limits and its required evidence. A person reviews every value and confirms. Nothing is stored until they do.
- Judges each change in Jira. A panel on the work item compares the described change with the plan and shows one of four verdicts: inside plan, outside plan, criteria failed, evidence missing. Every protocol check is listed with what was required, what was found and which file it came from.
- Uses files, not declarations. Results are read from the results file attached to the work item, not typed. Evidence counts only when a matching file is attached. Files are read again at every check.
- Blocks release. A Jira workflow validator refuses the transition unless the change is inside the plan and signed off, and says why. If it cannot read the verdict, it refuses.
- Binds sign-off to what was reviewed. A sign-off covers the exact plan, change and files. Change any of them and it no longer counts. Admins name approvers and can require that a second person signs.
- Notices when the plan moves. If the plan page is edited after it was confirmed, release closes for every change until an approver reviews the page and confirms the plan again.
- Keeps a record that shows tampering. Every event is linked to the one before it by hash. The register checks the chain on each load and exports it as CSV.
- Shows up in plain Jira. The verdict is a work item property, so
envelopeRelease = "blocked"works in search, filters, boards and dashboards. - Answers agents. A Rovo agent and an MCP server expose three read-only tools: what the plan allows, whether a proposed change would be inside it, and whether a work item may be released. Agents can ask. They cannot sign.
The rules are not specific to medicine. The app ships a second sample plan for a credit scoring model.
How we built it
Platform. Atlassian Forge on Node.js 24. One installation covers Jira and Confluence, so the plan confirmed in Confluence is the plan Jira checks against.
confluence:contentAction: "Use as change control plan" and the review dialogllm(Forge LLMs): Plan extraction with a forced tool call. The only model call in the appjira:issuePanel: Verdict panel on the work itemjira:workflowValidator: The release gatejira:adminPage,jira:globalPage: Approval rules; register and audit trailjira:entityProperty: Verdict searchable in JQLrovo:agent,rovo:mcp,action: Three read-only tools shared by the Rovo agent and outside MCP clientstrigger: Plan page updated
UI Kit throughout. Forge storage for the plan, changes, sign-offs, settings and audit entries. No external services and no data egress.
Three decisions carry the design.
- Rules decide. The model assists. The model proposes a plan once and a person confirms it. After that every verdict comes from one pure TypeScript module with no platform imports, covered by unit tests. The model never decides whether a change ships.
- Nothing is trusted from storage. The verdict is recomputed on every read from the plan, the described change and the files attached at that moment. One function decides whether a sign-off still counts, and the panel, the register, the agent and the gate all call it, so they cannot disagree.
- If in doubt, refuse. The gate fails closed. An unreadable verdict blocks the transition.
Bonus categories. Rovo agent: rovo:agent in manifest.yml, shown at 2:59 in the video. Atlassian MCP: rovo:mcp in manifest.yml exposing get-plan, check-change and get-work-item-status, with skills/envelope/SKILL.md for third-party agents, shown at 3:06 in the video.
Challenges we ran into
- The app could not see the plan page. The app's own identity could not read the Confluence space and received no page events there, so the first version of plan-edit detection missed an edit and let a change through. The check now runs live, as the acting person, at every panel load and at the gate.
- A sign-off that looked valid but was not. The panel once showed a sign-off as valid while the gate refused it, because two code paths decided separately. One function now decides for every surface.
- Scopes ruled out Bitbucket. A Jira and Confluence app cannot also hold Bitbucket workspace scopes, so the planned merge check became a Jira workflow validator.
- Two installs, two plans. Forge storage is per installation. Separate Jira and Confluence installs held different plans until the app declared both products in one installation.
- Names from an old plan. After the plan changed, values ticked under the earlier plan were kept and saved unseen, leaving a change "outside plan" with no visible cause. They are now shown and editable.
Accomplishments that we're proud of
The gate was tested against its own failure modes in the production environment on a live site, through the product UI:
- Unsigned change inside the plan, moved to Done: Refused, with the reason
- Same change after sign-off: Released
- Evidence file deleted after sign-off: Verdict dropped to evidence missing; refused
- Same file attached again: Still refused: the sign-off covered different files
- Plan page edited after confirmation: Release closed for every change until the plan was confirmed again
- Approver rule tightened: Existing sign-off stopped counting at once
- All three MCP tools called from Claude Code after OAuth: Correct results
97 unit tests cover the rules, sign-off validity, the gate decision, plan validation, the audit chain and extraction parsing, with no build step.
We also kept an honest list of limits. Evidence is matched by file name, not content. Detection of a tampered audit entry is unit tested but was not demonstrated by altering live storage. The sample plans are fictional and have not been reviewed by a regulatory professional.
What we learned
Most of the work was deciding what must not be trusted: a ticked box, a stored verdict, an old sign-off, a plan that has since been edited. Each of those was a real bug or a near miss before it was a principle.
The useful place for a language model here was narrow. Turning a page of prose into structured rules is a good job for it, as long as a person confirms the result. Deciding whether a regulated change ships is not.
Exposing the same rules to agents cost little once they were plain actions, and it changes when the question gets asked: a coding agent can check a proposed change against the plan before any work is done, not at release.
What's next for Envelope
- Check evidence by content, not file name.
- Read results directly from a CI run instead of an attachment.
- Export a signed change record to Confluence for the quality file.
- Review of the plan structure by regulatory professionals, and a pilot with a team that holds an authorized plan.
- Marketplace listing.
Built With
- atlassian-forge
- confluence
- javascript
- jira
- mcp
- node.js
- react
- rovo
- typescript