Sourced

A fact-check desk for newsroom drafts in Confluence: every claim, its evidence, and a warning when a verified sentence changes.

Comment

Inspiration

Many newsrooms write and edit stories in shared documents, and Confluence is one of them. The fact-check usually lives somewhere else: in the checker's notes, in email and in chat. Two things go wrong. A claim gets waved through on weak evidence, such as another outlet's report or the reporter's own notes. And a sentence that was checked on Tuesday is edited on Wednesday, so the published number is not the number anyone verified. Sourced keeps the claims, their evidence and their status on the draft itself, and it watches the draft for changes.

What it does

  • Lists the claims in a draft. Open a draft and ask the Sourced agent in Rovo to fact-check it. The agent reads the draft, including the links the writer cited. It lists every checkable claim with:
    • the exact words from the draft;
    • its type: number, direct quote, date, attribution, comparison or anonymous source;
    • the evidence the writer already gave.

In our demo it found 9 claims in a story about a city council vote. Nothing is saved until you confirm in Rovo's review card, and every quote must appear word for word in the draft.

  • Applies the newsroom's standard, with a reason. Deterministic rules decide each claim's status: Verified, Needs evidence, Needs second source, Needs editor, or Changed since verified. Every status comes with a sentence you can check, for example "Comparison needs a primary document that supports the comparison. Evidence so far: another outlet's report. Another outlet's report does not verify a claim on its own." The default standard:
    • numbers need a primary document;
    • direct quotes need a recording, a transcript or the speaker's confirmation;
    • comparisons need a document and then an editor's sign-off;
    • anonymous claims need two independent sources and then an editor;
    • another outlet's report, or the reporter's notes alone, never verify a claim.
  • Catches edits after the check. A Confluence page-update trigger re-reads the draft. A verified sentence that no longer appears becomes "Changed since verified". An unverified sentence that disappears is marked removed. In the demo, the writer changed "$48 million" to "$52 million", and the macro and the byline flagged it at once. The agent then re-checked the claim with the new wording and the revised budget table.
  • Status on the page. A "Fact-check status" macro lists every claim, blockers first, with its reason and evidence. It has two forms:
    • Add evidence, with an editor's sign-off;
    • Add a claim, for teams without Rovo.

The page byline shows "Fact-check 5/9 ยท 1 changed".

  • The desk. A Confluence global page, "Fact-check desk", shows:
    • every draft in fact-check with its readiness;
    • claims by status;
    • which drafts are ready to publish;
    • the claims that still block publication.
  • A fact-check log. The agent writes the log as a page comment: every claim with its status, evidence and reason, so the record stays with the story.

How we built it

  • Forge, Confluence only. One app. Modules:
    • rovo:agent with six actions: get-draft, record-claims (CREATE), list-claims, update-claim (UPDATE), desk-summary and write-report (CREATE);
    • a Confluence macro;
    • confluence:contentBylineItem with dynamicProperties;
    • confluence:globalPage;
    • a trigger on avi:confluence:updated:page;
    • Forge Custom Entities, indexed by page and by status plus update time.

The UI is UI Kit, with a UI Kit donut chart.

  • AI where it helps, code where it must be right. The agent reads the prose and finds the claims and the cited evidence. Statuses, reasons and change detection are plain code covered by 11 offline tests. The tests cover the rules for each claim type, independent sources, quote matching that ignores curly quotes and dashes, change detection, and parsing of the agent's input.
  • Guardrails for the agent. Each recorded quote is checked against the draft text, and quotes that are not found are sent back. Links are stored only when they are real http(s) links. During testing the agent once proposed a made-up mailto link. Sourced did not store it, and we tightened the prompt.
  • Permission-aware. Reads for people run as the current user, and the desk only lists drafts the viewer can open. The trigger reads the page as the app, only to compare text. The fact-check log is written as the user.
  • No external calls. After the production deployment, the Forge CLI reported that the version is eligible for the Runs on Atlassian program.
  • Measured on a live site. We ran the whole flow on the production app on a demo site with fictional data:
    • claim extraction, the review card and recording;
    • evidence updates and an editor's sign-off;
    • change detection after an edit, and a re-check with the new wording;
    • the desk, the byline, the macro forms and the fact-check log.
  • Demo video. Screen captures of the production app. The people, places and documents are fictional. The narration is synthetic (Kokoro-82M text to speech, Apache-2.0).

Challenges we ran into

  • The agent sometimes split one fact into pieces, such as the day of a vote and the vote itself. We added a rule to the prompt: one claim per fact, with the qualifying words kept in the quote.
  • Trashed Confluence pages still answer the page API. The desk briefly counted a trashed draft, so Sourced now treats only current pages as drafts in fact-check.
  • A selected-text context menu was hard to make reliable in our test browser. We replaced it with an "Add a claim" form in the macro, which we could test end to end.

Accomplishments that we're proud of

  • The rules never let another outlet's report verify a claim, and every status says why.
  • A changed sentence cannot slip through: the byline and the macro flag it as soon as the page is saved.
  • The fact-check record lives on the story, not in someone's notebook.

What we learned

  • Fact-checking is mostly bookkeeping around judgment. The agent is good at finding claims, and the rules are good at saying what is still missing.
  • Showing the reason with each status matters more than the status itself.

What's next

  • House rules per space, such as a stricter standard for investigations.
  • Embargo and publish-time reminders for drafts that are not ready.
  • Exporting the fact-check log with the published story.

How it meets the technology requirement

Sourced is built entirely on Atlassian Forge. It integrates Confluence (a macro, a byline item, a global page, a page-update trigger, page reads and comments) and Rovo (an agent with six actions). It makes no external calls.

Bonus prize: Marketplace published app

  • On 14 November 2026 we submitted Sourced for listing on the Atlassian Marketplace as Sourced - Fact-Check Desk for Confluence (free, Confluence only, Marketplace app 2054618358). It is pending approval. Review ticket: https://ecosystem.atlassian.net/browse/ECOHELP-185622
  • The listed build is the same code under a separate Forge app ID, because a Forge app that is submitted for listing can no longer be shared by installation link, and judges need the link.

Privacy and support

  • Sourced runs entirely on Atlassian Forge and makes no external calls.
  • It stores the following in Forge storage on the customer's site:
    • the claims' exact words, types and statuses;
    • evidence notes and links;
    • the page ID, title and link.

It does not store account IDs, people's names or email addresses.

  • Questions or problems: leave a comment on this Devpost page.
Share this project:

Updates

Submission history