Inspiration Every team knows the moment: a fresh Jira issue lands with a title like "It's broken", no version, no steps — and a senior engineer burns twenty minutes just figuring out what it is, how bad it is, and whose problem it should become. Triage is the least loved, most repeated ritual in software teams, and it is almost entirely pattern-matching over text. We wanted to know: how much of triage can be done deterministically — fast, free, and auditable — before any AI needs to be involved?
What it does The moment a Jira issue is created, TriagePilot: (1) Classifies the type — bug, feature, task or support — from keyword evidence, the reporter's issue type, question shape, and objective signals like stack traces and HTTP error codes. (2) Scores the urgency — critical / high / medium / low — from weighted impact signals (outage, data loss, production, deadlines) minus "minor / no rush" signals. Every point is recorded as a human-readable reason. (3) Suggests an assignee by matching the issue's component and topics against the team's resolution history, with confidence and runners-up. (4) Drafts the first response — a triage summary plus reproduction steps for bugs, an acceptance checklist for features, a handling outline for support, and an explicit "information needed" list when the report is thin. You can consume that four ways: the auto-posted comment on the new issue, the issue panel (with one-click apply of comment, labels, priority and assignee), the TriagePilot Rovo agent in chat, and — for teams building their own agents — as MCP tools in Rovo Studio (read-only by design).
How we built it A pure, dependency-free rules engine (src/triage, ~700 lines) with 30 unit tests over 13 fixture issues, runnable with Node's built-in test runner. The engine alone decides labels and emits the evidence for each decision. A Forge app around it: a trigger on avi:jira:created:issue, a Custom UI jira:issuePanel (plain HTML/JS + @forge/bridge, bundled with esbuild), a @forge/resolver backend, and Forge Storage (KVS) caching the trigger's result so the panel renders instantly. A Rovo agent (rovo:agent, prompt as a manifest resource) wired to three actions (triage-issue, suggest-assignee, post-triage-comment). The agent's prompt forbids it from inventing labels: it presents the engine's output and only narrates around it. A rovo:mcp module exposing the two GET actions as tools, plus an open-source skill.md playbook describing the triage workflow. A small Markdown-to-ADF converter so drafts post as native Jira comments.
Challenges Keeping the LLM honest: the interesting design problem was the opposite of most AI projects — the agent had to not freestyle. We solved it by making the deterministic engine the only source of labels and writing the agent prompt as a presentation contract over action outputs. Real-world report quality: vague issues ("It's broken") are the norm, not the exception; the engine flags them (vague-summary, no-description, possible-duplicate) and generates an "information needed" list instead of pretending to know more than it does. Forge platform details: module keys and payload shapes differ subtly from older blog posts (the action module is "action", not "rovo:action"); we verified every manifest key against the current Atlassian developer docs.
What we learned Deterministic-first design is a feature, not a compromise: triage labels that come with receipts ("critical, because: production down + payments failing + stack trace") get trusted and used, and the LLM becomes far more useful when its job is explanation rather than judgement.
What's next Learn the component history automatically from resolved issues instead of a hand-maintained map. Pull related Confluence pages into the draft via the Teamwork Graph (scopes are already declared). Per-project calibration UI in the panel. Marketplace listing so any Jira team can install it.
Try it Install TriagePilot (production build) on any Jira Cloud site: https://developer.atlassian.com/console/install/7993a295-000a-4fa8-bd14-6a997848965a?signature=AYABeDFCsKNlm7mDGtPxE2gzTHAAAAADAAdhd3Mta21zAEthcm46YXdzOmttczp1cy13ZXN0LTI6NzA5NTg3ODM1MjQzOmtleS83MDVlZDY3MC1mNTdjLTQxYjUtOWY5Yi1lM2YyZGNjMTQ2ZTcAuAECAQB4CxqLTo9l4lqXuFjNim1xJM%2Bp%2F3TVP9xHS2BrU5DRRucBKQpUkwfqZdI7EFSfGENkKQAAAH4wfAYJKoZIhvcNAQcGoG8wbQIBADBoBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDP5GNWU1bC%2BMmRx2IAIBEIA7Hn566SWuqrKsZNnK8Zp0ul5hJrqNTs1LbbKmyImI0AcQ5%2FjYBTkHoodPkrLOhsBF98LZkQ8qX4LJSGwAB2F3YBTkHoodPkrLOhsBF98LZkQ8qX4LJSGwAB2F3cy1rbXMAS2Fybjphd3czt1ZXN0LTI6NzA5NTg3ODM1MjQzOmtleS80NjBjZTZiLTAwYzMtNGRlMi04NzdiLTYyN2UyMDYwZTVjYwC4AQICAHi9bVMAkme4qVQX1fQHo6ePw3H40A7cGfz7yrmSfBWMXgGVe1MfiSR%2BMfLFmXHgk1pxAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMyIc5OuM5HDZbphYDAgEQgDu7Zi496KuZRLZ9hOPyIOYA5RA8qYDfkoc61vyarb1z%2FJra1wZHMkME1F4r6voy7CylMmF1ct0%2FZ4HDGAAHYXdzLWttcwBLYXJuOmF3czprbXM6dXMtZWFzdC0xOjcwOTU4NzgzNTI0MzprZXkvNmMxMjBiYTAtNGNkNS00OTg1LWI4MmUtNDBhMDQ5NTJjYzU3ALgBAgIAeDYASx0XCjkny4iLV6vz%2BKtEGSGT6kdt0QRpkn6D%2FlVEATDTTm1mgXfg9ZHez5VrFiYAAAB%2BMHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAwWpypttKZ5TL28UpQCARCAO4lSFnUIwAnCVyQ%2BbTRJsUDLnbEuitaIFj3QfvjfI%2BVnZdEkHnox2n4mmoKfTjt5AUklAgDy12c2uppCAgAAAAAMAAAQAAAAAAAAAAAAAAAAANjVSF0p009%2B1eiyY8A3S8%2F%2F%2F%2F%2FAAAAAQAAAAAAAAAAAAAAAQAAADLGXRu%2B3qeVlnIMSks3S7EjdEWC0fb1aZsWOU256jQShbQQEQS69atbrpFfdp9%2BS1gvINH2h4XYnTZgN%2FU3R2xpzTE%3D&product=jira Live demo site used in the video: https://triagepilot-dev.atlassian.net
