RealityCheck

Traceability Analysis
Triage Chart
Findings and Claims
Visualizations
Landing Page

Inspiration: Beyond What Documents Claim What it does - The Story Behind RealityCheck

I have been working in corporate since the last 9.5 years and I have often found that technical projects generate enormous amounts of documentation—requirements, implementation notes, configuration records, test reports, and observations. Yet, an important question often remained unanswered: Does the available evidence actually prove what the documentation claims?

A requirement described some expected behavior but many a time I had no idea if it was being implemented properly. An implementation note did not prove that something was tested. A configuration snapshot did not necessarily reflect the deployed system.

Connecting these pieces manually across multiple documents was time-consuming and many a time, inconsistencies and verification gaps were overlooked. That inspired me to build 'RealityCheck'—a platform that goes beyond document summarization to examining the relationship between what is claimed and what the actual evidence says.

How I Built It:

I designed RealityCheck as an AI-powered, evidence-grounded analysis pipeline.

Document ingestion: Supports PDF and UTF-8 text files, using Docling for PDF extraction.
Claim extraction: Identifies meaningful claims, classifies their roles, and preserves their source context and provenance.
Evidence reconciliation: Uses NVIDIA Nemotron models through Nebius Token Factory to evaluate claims against available evidence.
Finding generation: Identifies verified claims, reasonable inferences, uncertain outcomes, and actual discrepancies.
Multi-document analysis: Compares evidence across multiple documents to uncover contradictions and dependencies that may not be apparent when reviewing documents individually.
Visualization: Presents findings and their supporting evidence through an interactive React and TypeScript interface, backed by a Python FastAPI application.

I deliberately separated claim extraction from deeper evidence reconciliation, using different models for these stages:

Nemotron 3 Nano 30B A3B for structured claim extraction.
Nemotron 3 Super 120B A12B for evidence reconciliation and reasoning.

Challenges I Tackled:

Teaching the system to distinguish between what is intended, what is documented, and what has actually been observed. e.g. an expected test outcome must never be interpreted as a successful test. Similarly, missing execution evidence should indicate an unverified requirement—not automatically imply implementation failure.
Extracting meaningful information from technical documents with varied layouts and formatting.
Preserving traceability between extracted claims, source documents, and supporting evidence.
Avoiding unsupported conclusions and duplicate findings.
Identifying meaningful cross-document inconsistencies without inventing relationships.

These challenges pushed me to focus not just on generating AI responses, but on making every finding explainable and traceable.

What I Learnt:

Building RealityCheck reinforced an important lesson: AI-generated conclusions are only as useful as the evidence supporting them. = I learnt the importance of separating extraction from reasoning, explicitly defining evidence roles, controlling unsupported inferences, and preserving provenance throughout the analysis pipeline.
Working with Nebius Token Factory and NVIDIA Nemotron models also gave me practical experience integrating different models into a real application, assigning them distinct responsibilities, and evaluating their usefulness in a complex reasoning workflow. The free credits provided by Nebius were particularly valuable in enabling me to experiment, iterate, and move from an idea to a working implementation.

What's Next?

RealityCheck is a foundation that can be extended with additional document formats, stronger extraction capabilities for complex visual documents, deeper observability, and integrations with engineering and enterprise systems.
My larger vision is simple: Don't just ask what the documents say. Ask what the evidence actually proves. That's RealityCheck!!!