For Sure?
Inspiration
Misinformation has always spread. But AI has fundamentally changed its scale, speed, and credibility.
A study published in December 2025 by Next in partnership with Médiamétrie tracked 251 AI-generated fake news sites over ten months. The results were striking: these sites collectively reached 15.7 million unique monthly visitors, placing them at a level comparable to a top 10 news outlet in France. Even more alarming, people over 50 visited more AI-generated fake news sites than real news sites, and 77% of that traffic came directly from Google.
This is no longer a fringe problem. It is a democratic emergency. When a significant share of a population's news consumption is algorithmically steered toward AI-generated content designed to mimic journalism, the foundations of informed civic life are under direct threat. As specialists in information put it, this represents a double threat: it erodes public trust in media, while economically weakening the traditional newsrooms that invest in investigation, source verification, and editorial accountability.
And this is only the beginning. LLMs and agentic AI systems are moving toward continuous learning, models that update on the fly on new information. This makes them more adaptable, but also dangerously exposed: a single corrupted claim ingested at the wrong moment can propagate through an entire pipeline before anyone notices. The speed of misinformation now vastly outpaces our defensive infrastructure.
That asymmetry is exactly what BlueDot Impact's philosophy of defensive acceleration is built to address. We realized that the solution isn't just fact-checking for humans — it's building a verification layer directly into the AI stack.
Beyond the urgency, we also saw a market. The global AI market reached $244 billion in 2025 and is projected to hit $827 billion by 2030. Within that, AI governance, the infrastructure of trust, oversight, and accountability — is growing even faster: the AI governance market is projected to reach $7.38 billion by 2030, at a 51% CAGR from 2025. Every enterprise deploying agentic AI will need to answer one question: can I trust what my agent just learned? For Sure? is the answer.
What it does
For Sure? is a real-time information provenance pipeline for Bluesky. Given any post, it automatically:
- Traces the origin of the claim by crawling backwards through replies, reposts, and the author's liked posts to find earlier versions of the same information
- Measures distortion at each hop using an LLM-powered comparator that scores how much a claim has drifted from its source, producing a
distortion_score$\in [0, 1]$ where $0$ means identical and $1$ means completely deformed - Builds a provenance graph where nodes are posts and edges carry fidelity scores, allowing anyone to visually trace how a piece of information was transformed as it spread
- Synthesizes a verdict , a plain-language summary of whether the claim is reliable, where it originated, and how it mutated along the way
The result is served through a Ruby on Rails interface as an interactive graph, making the full audit trail accessible to journalists, researchers, and AI systems alike.
How we built it
The pipeline is composed of 7 chained steps:
| Step | Role |
|---|---|
| 1 — Fetcher | Takes a Bluesky URL, fetches the post via AT Protocol API: text, date, author, engagement metrics. This is the root node at time T |
| 2 — Keywords | Passes the post to an LLM to extract entities, claimed facts, and search keywords as structured JSON |
| 3 — Searcher | Searches Bluesky for posts containing those keywords in a time window before T, filtered by minimum engagement |
| 4 — Validator | For each candidate, asks the LLM "do these two posts discuss the same fact?" — keeps only those above a 0.7 similarity threshold |
| 5 — Analyzer | For each validated pair, runs the main LLM prompt to produce info_lost, info_added, distortion_score, tone_shift, and semantic_similarity |
| 6 — Provenance Graph | Creates one node per post, one directed edge per validated pair, with post metadata on nodes and analysis results on edges (NetworkX) |
| 7 — Source Scoring | When a post has no prior antecedent, scores its reliability via heuristics: account age, followers/following ratio, known media domain links |
| 8 — Quality Propagation | Traverses the graph from source to root: each node inherits its parent's score weighted by the edge's distortion — $\text{score} = \text{score}_{parent} \times (1 - \text{distortion_score})$ |
| 9 — Synthesis | Passes the full chronological chain to the LLM one final time: original fact, what was lost/added/distorted, and a clean factual summary |
| 10 — Visualization | Renders the graph with D3.js, nodes colored green to red by quality score |
The backend is in Python with atproto for Bluesky API access and Gemini 2.5 Flash for all LLM steps. The frontend is a Ruby on Rails app rendering the provenance graph interactively.
Source trust is scored via heuristics (account age, followers/following ratio, known media domains), then propagated through the graph using:
$$\text{score}{node} = \text{score}{parent} \times (1 - \text{distortion_score})$$
If a node is identified as a fact-check by the LLM, it boosts the scores of downstream nodes.
Challenges we ran into
The Bluesky API is more restrictive than expected. Some endpoints behave inconsistently — for instance, fetching a user's follows can silently fail or return partial results depending on account settings. We had to build fallback logic at each step so the pipeline degrades gracefully rather than crashing.
Relevance vs. recall in step 3. Finding posts that are genuinely prior sources, not just thematically similar, required careful tuning of the search strategies. Keyword search casts a wide net but brings noise; The followees strategy is more targeted but depends on the author's social graph.
LLM consistency at scale. Running Gemini in parallel across many candidate pairs introduced variance in similarity scores. We mitigated this with temperature=0.0 and structured JSON outputs, but edge cases in parsing required robust fallback handling throughout.
Exponential graph depth. With max_depth=4, the number of API calls grows exponentially. We had to balance graph richness against runtime, ultimately settling on sensible defaults and exposing the key parameters to the user.
Accomplishments that we're proud of
- A fully working end-to-end pipeline that takes a single Bluesky URL and produces a verifiable provenance graph in minutes
- A distortion scoring system grounded in actual semantic comparison, not just keyword overlap
- A graceful multi-strategy search that never hard-fails — if one source is unavailable, the pipeline continues
- Making something genuinely useful for AI safety: a tool that could act as a verification layer for any agentic system ingesting social media content
What we learned
Building this taught us that information provenance is fundamentally a graph problem, not a search problem. The instinct is to look for "the source", a single node. The reality is a web of partial transmissions, each introducing small distortions that compound.
We also learned that the Bluesky AT Protocol, despite being newer, offers richer provenance metadata than Twitter ever did, replies, reposts, and quote posts all carry explicit parent references, making the lineage much easier to reconstruct programmatically.
On the LLM side, structured outputs with temperature=0 are non-negotiable when you need consistent scores across hundreds of calls. Any variance compounds across pipeline steps.
What's next for For Sure?
- Live monitoring mode, watch a claim in real time as it spreads, alerting when distortion crosses a threshold
- Multi-platform support, extending beyond Bluesky to Mastodon and Reddit using the same graph architecture
- Agent API, a simple REST endpoint so any LLM agent can call For Sure? before acting on a claim, receiving a trust score and corrected version if needed
- Collaborative trust database, a crowd-sourced layer where verified journalists and researchers can flag known misinformation sources, feeding back into the quality scores
Log in or sign up for Devpost to join the conversation.