Inspiration
Every engineering organisation has the same story. A project slips or causes an incident, the retro captures exactly what to do differently, and six months later another team repeats it. The knowledge already exists in Jira. It just isn't in front of the team at the moment they start new work.
What it does
When an epic is created, Hindsight finds the most similar finished work on the site and measures what actually happened to it: planned vs actual time, schedule slip, bugs, incidents and who fixed them, and the retro notes. Seconds later it comments on the new epic with an evidence-based pre-mortem:
- Risk level and predicted slip, computed from real outcomes, with a confidence level
- What happened last time: the comparable past projects, with links to their incidents
- Lessons → tasks: one click turns lessons from past retros into Jira tasks under the epic
- People who've done this before: who led similar work or fixed its incidents
- A learning loop: when the epic is done, Hindsight scores its own forecast and tracks accuracy
The same intelligence is available as a Rovo agent ("What went wrong last time we touched payments?") and through an MCP server, so AI assistants such as Claude, Cursor or Codex can check the company's history before writing code.
How we built it
A Forge app for Jira that runs on Atlassian (no external servers):
- Jira surfaces:
jira:issuePanelandjira:projectPage(UI Kit), plustriggermodules on issue created/updated for the automatic pre-mortem and the accuracy loop - Rovo:
rovo:agent"Hindsight" with a reusablerovo:skill(hindsight-premortem) and 4actionmodules (find similar work, get forecast, get experts, add lessons as tasks) that other agents can invoke - MCP:
rovo:mcpexposes the read-only tools (find-similar-work, get-forecast, get-experts) through Atlassian's MCP server; verified end to end from an external client - Teamwork Graph: Teamwork Graph read scopes for users and teams, a persisted Cypher query to enrich experts with their Atlassian teams, and a
graph:connectorthat writes each forecast into the graph as an object associated with the epic and the past work it cites (active once a site admin adds the connection) - AI: Forge LLMs (
@forge/llm, Atlassian-hosted Claude) re-ranks candidates and writes the lessons. Every number is computed in code from Jira data, every claim must cite a real issue key, and anything the model invents is dropped. If the model is unavailable, Hindsight still produces a full data-only forecast - Matching: TF-IDF similarity with business-domain filtering, so payments work is compared with payments work rather than any project containing the word "migration"
- Storage: Forge KVS for forecasts, added-task tracking and accuracy
Challenges we ran into
- Making the forecast trustworthy. Our first version let the model pick an unrelated "OAuth2 migration" as evidence for a payments epic. We fixed this in code with domain filtering and by never letting the model produce numbers.
- Testing on an empty site. Judges' sites have no history, so we built a one-click sample-history loader that creates 12 real finished epics with child issues, incidents and retro comments.
- Rovo on developer sites. Free developer sites don't include Rovo chat, so we tested the agent on a Forge demo site.
Accomplishments that we're proud of
- A complete loop inside Jira: detect → explain with evidence → act (tasks) → learn (accuracy)
- The same forecast from three surfaces: the Jira panel, a Rovo agent and an MCP server
- Runs on Atlassian, with a deterministic fallback when the AI is unavailable
What we learned
Teams don't need more dashboards. They need the right piece of history at the moment they make a decision, with evidence they can click.
What's next for Hindsight
Confluence retro pages and JSM incidents as more memory sources, Bitbucket pull-request checks, per-team calibration from the accuracy loop, and a Marketplace listing.
Built With
- atlassian-forge
- claude
- javascript
- jira
- mcp
- node.js
- react
- rovo
- teamwork-graph