EvidenceMesh: Autonomous Causal Verification Engine

Autonomous Causal Verification & Cryptographic Proof Engine: Turning Generative AI from ungrounded assertions into verifiable DAGs and mathematically calibrated evidence proofs.

Comment

💡 Inspiration

"Don’t pitch the future. Build evidence."

Generative AI models produce remarkably articulate, confident statements. However, in safety-critical sectors—such as clinical pharmacology, clean energy storage, hardware engineering, and cybersecurity—an unverified assertion or statistical hallucination can cause catastrophic failures.

We were inspired by a fundamental observation: human experts do not evaluate claims based on conversational fluency; they require a falsifiable causal chain, empirical grounding, calibrated uncertainty bounds, and tamper-evident audit trails.

We built EvidenceMesh to transform AI from an opaque, conversational chat box into an autonomous, mathematically rigorous, and cryptographically provable causal evidence verification engine.

⚡ What it Does

EvidenceMesh ingests complex multi-sentence assertions, scientific reports, or technical specifications and runs an autonomous multi-agent verification pipeline:

  1. Atomic Proposition Decomposition: Automatically decomposes compound sentences into discrete, falsifiable predicate tuples $(Subject, Predicate, Object, Number, Unit, Condition)$.
  2. Causal Claim Dependency Graph (DAG): Constructs an interactive Directed Acyclic Graph resolving prerequisite proofs, causal mechanisms, and cross-claim entailments.
  3. Multi-Agent Adversarial Cross-Examination (Red-Teaming): Deploys an Extractor Agent against an Adversarial Red-Teamer that actively searches for hidden contradictions, physical impossibilities, and clinical contraindications.
  4. Bayesian Epistemic Uncertainty Calibration: Replaces arbitrary confidence percentages with exact conjugate Beta-Binomial updating, computing $\pm 1.96\sigma$ (95%) credible intervals, Expected Calibration Error (ECE), and Brier reliability scores.
  5. Cryptographic Merkle Proof Ledger: Hashes every verified claim and citation into an immutable SHA-256 Merkle Evidence Tree, issuing verifiable JSON-LD Proof Certificates with deterministic tamper detection.
  6. Interactive Glassbox Console UI: Provides real-time HTML5 Canvas physics-based DAG visualization, live agent stream timelines, and one-click benchmark scenarios.

🔬 Mathematical Rigor & Uncertainty Formulation

1. Bayesian Conjugate Beta-Binomial Updating

Given prior skepticism parameters $\alpha_0, \beta_0$, observed positive evidence weights $w_i \cdot c_i$, and refuting evidence weights $w_j \cdot (1 - c_j)$, the posterior belief distribution follows:

$$\alpha_{\text{post}} = \alpha_0 + \sum_{i=1}^{N_{\text{pos}}} w_i \cdot c_i, \quad \beta_{\text{post}} = \beta_0 + \sum_{j=1}^{N_{\text{neg}}} w_j \cdot (1 - c_j + 0.5)$$

$$\mathbb{E}[\theta] = \frac{\alpha_{\text{post}}}{\alpha_{\text{post}} + \beta_{\text{post}}}$$

$$\text{Var}(\theta) = \frac{\alpha_{\text{post}} \cdot \beta_{\text{post}}}{(\alpha_{\text{post}} + \beta_{\text{post}})^2 (\alpha_{\text{post}} + \beta_{\text{post}} + 1)}$$

$$95\% \text{ Credible Interval} = \left[ \mathbb{E}[\theta] - 1.96 \sqrt{\text{Var}(\theta)}, \; \mathbb{E}[\theta] + 1.96 \sqrt{\text{Var}(\theta)} \right]$$

2. Expected Calibration Error (ECE)

To guarantee that confidence scores reflect true accuracy and penalize overconfidence:

$$\text{ECE} = \sum_{m=1}^M \frac{|B_m|}{N} \left| \text{acc}(B_m) - \text{conf}(B_m) \right|$$

$$\text{Brier Score} = \frac{1}{N} \sum_{i=1}^N (p_i - y_i)^2$$

3. Cryptographic Merkle Root Proofs

Every atomic claim is hashed into an immutable leaf node with its semantic citations:

$$h_i = \text{SHA256}(\text{Domain} \,||\, \text{ClaimID} \,||\, \text{Status} \,||\, \text{Confidence} \,||\, \text{Citations})$$ $$h_{\text{root}} = \text{MerkleTree}({h_1, h_2, \dots, h_k})$$

🛠️ How We Built It

  • Backend & Verification Engine: Engineered in Python 3.11 with FastAPI and Pydantic V2, utilizing NumPy and SciPy for numerical Bayesian posterior integration and Tarjan's DFS for DAG cycle detection.
  • Multi-Agent Adversarial Swarm: Asynchronous consensus orchestrator coordinating Decomposer, Grounding Verifier, Red-Teamer, and Synthesizer agents.
  • Cryptographic Ledger: Custom SHA-256 Merkle Evidence Tree engine producing verifiable JSON-LD certificates and cryptographic signatures.
  • Glassbox Web UI: Built with modern dark-mode Tailwind CSS, dynamic HTML5 Canvas force-directed DAG rendering, and real-time WebSockets streaming.
  • Testing & Quality Assurance: 16 comprehensive pytest unit and integration tests passing with 100% reliability in under 0.35 seconds.

🚧 Challenges We Ran Into

  1. Topological Claim Dependencies & Cycle Prevention: Parsing natural language into valid DAGs often results in circular assertions. We implemented cycle-detection algorithms to decouple logical entailments from contradiction edges, ensuring robust topological sorting.
  2. Deterministic Calibration Under Sparse Evidence: When verifying novel claims with sparse citations, standard MLE overfits. Developing the conjugate Beta prior framework allowed us to quantify epistemic vs. aleatoric uncertainty smoothly.
  3. Cryptographic Reproducibility: Ensuring Merkle leaf serialization is strictly canonical across different environments and platforms required deterministic JSON sorting and domain salting.

🏆 Accomplishments We're Proud Of

  • Zero-Hallucination Causal Pipeline: Successfully catching critical life-safety contradictions across diverse real-world benchmarks (such as detecting fatal penicillin allergy cross-reactivity or unmanaged solid-state battery thermal limits).
  • 100% Test Pass Rate: 16/16 automated pytest unit and integration tests verifying every algorithmic subsystem.
  • Tamper-Evident Proof Certificates: Delivering self-contained JSON-LD certificates that can be audited independently using cryptographic Merkle roots.
  • Zero-Config Local Deployment: A single python run.py command launches the entire stack and opens the UI in your browser.

📚 What We Learned

  • Decomposition Prevents Hallucination Drift: Splitting complex compound claims into atomic predicate tuples isolates falsifiable facts without losing contextual relationships.
  • Adversarial Red-Teaming is Essential: Standard LLM prompts agree with the user by default; an active Adversarial Cross-Examiner is required to uncover hidden assumptions and physical contraindications.

🚀 What's Next for EvidenceMesh

  • Live MCP (Model Context Protocol) Connectors: Expanding grounding connectors to live scientific repositories (PubMed, arXiv, IEEE Xplore, and SEC EDGAR).
  • Zero-Knowledge Evidence Proofs (zk-SNARKs): Enabling private verification where organizations can cryptographically prove compliance without disclosing proprietary data.

Built With

Share this project:

Updates

Submission history