Inspiration

Every release has a moment where someone asks "are we good to ship?" and the answer lives in five places: the Jira version, the pull requests that were supposed to close it, the deployment that went out on Thursday, the review comment nobody answered, and one person's memory of all four. That question becomes a twenty-minute checklist, and the checklist is different depending on who runs it.

AI assistants made this worse before they made it better. Ask one "is 2.0.0 safe to ship?" and it will happily summarize the Jira version and call it ready, because it cannot see that two work items in scope are still open and that no pull requests or deployments were ever linked. It has no way to say "I could not read that." That gap — confident answers built on data that was never there — is what I wanted to fix.

What it does

Ship Confidence answers one question per release and returns four things: a verdict, a score, the specific reasons, and an honest statement of what it could not read.

The verdict is one of safe, safe with risk, not ready, or unknown. A version with two open work items in scope comes back not ready, 59/100, with both blockers named and a single next action: clear the first one, then refresh. Not a list of forty signals — the one thing standing in the way.

It ships as two surfaces over one domain layer. In Jira, a project page renders the same answer with no AI involved, so a release manager who never opens a chat client still gets it. Through MCP, six read-only tools expose the same logic to Rovo or any MCP client: readiness, open blockers, risk factors, an explanation of a single blocker, drafted release notes, and a comparison between two releases.

The part most apps get wrong

The interesting problem was not the scoring. It was distinguishing "everything is fine" from "I could not see everything."

Jira can tell you a work item is open. It cannot tell you whether that work item has a pull request, a deployment, or a review thread — that lives in the Teamwork Graph, and only if the site has connected its SCM and deployment integrations. A tool that treats absent data as good news is actively dangerous at a release gate.

So every answer carries a coverage map with per-source statuses: read, empty, not permitted, or unsupported. unverified means unknown, never "probably fine." When the graph cannot answer, the app says what it could not read, in the same breath as the verdict rather than buried underneath it.

How I built it

One domain layer, two surfaces, and a hard boundary between them:

  • src/domain/ — pure scoring and comparison. No SDK imports, no clock, no I/O, so every rule is testable and the answer is reproducible.
  • src/graph/ — curated GraphQL queries, a client port, a Forge adapter, a loader that resolves work items to pull requests and deployments, coverage tracking, and a probe.
  • src/tools/ — payload parsing, a shared result envelope, and one pipeline every tool goes through.

The Jira page is @forge/react; the tools are six rovo:action modules (actionVerb: GET, so read-only by construction) listed in a rovo:mcp module, so Atlassian handles discovery, OAuth and protocol negotiation. A rovo:skill ships with the app and carries the judgement the tools cannot: which tool answers which question, the answer shape, and the rule that a coverage caveat comes before any reassurance.

Challenges

GraphQL argument shapes. The published SDL per site is the only reliable source of truth, and version-scoped traversal is not available everywhere. I added a probe rather than guessing, and ended up reading release scope from Jira search where the graph could not filter it — with the graph still supplying the evidence Jira cannot see.

An app that looks broken. MCP tools ship disabled. A judge who installs and asks a question gets silence, and concludes the app does not work. That is a documentation problem as much as a permissions one, so the setup path is stated every place someone might land: the README, a setup guide, and the project page itself.

Honesty under pressure. The temptation is always to round "unknown" up to "probably ready," because that is what a person wants to hear before a release. Refusing that is a design decision, and it is the one I am most confident about.

What I learned

  • Missing data is a first-class answer, not an edge case. Modeling it explicitly changed the scoring, the UI and the tool descriptions at once.
  • Tool descriptions are prompts. Writing them for selection — each stating the question it answers and its boundary — matters more than documenting parameters.
  • Read-only is a feature. Because there is no write path anywhere, a team can point an agent at a release without worrying what it might change, and identity always comes from the Forge context rather than from the model's input.

Try it

Install from the link on this page (Jira only), then have a site admin enable the tools under Administration → Apps → Sites → Connected apps. The demo site is the SHIP project, version 2.0.0. On a site with no data every tool still answers — it reports unverified and names what it could not read, which is the intended behaviour, not an error.

Built With

Share this project:

Updates

Submission history