EvidenceWeave

A local-first evidence cross-checking workspace that helps criminal defense teams detect inconsistencies, trace every AI flag to its source, and keep final judgment with lawyers.

Comment

Inspiration

Criminal case files often contain multiple statements, witness testimony, objective records, and repeated interviews collected at different times. Important inconsistencies may be scattered across dozens of pages and different documents.

We wanted to explore whether AI could help lawyers locate relationships that deserve closer review without asking the model to make legal judgments for them.

That led to EvidenceWeave: a local-first evidence cross-checking workspace built around one principle:

No Citation, No Claim.

Every AI-generated flag should remain traceable to the original material, while the final decision stays with the lawyer.

What it does

EvidenceWeave imports case documents and organizes evidence around specific people and factual issues.

It extracts atomic facts from source materials and compares potentially related facts across defendant statements, witness testimony, surveillance records, ride-hailing records, and other documents.

Potential relationships are presented in a consistency matrix, including:

  • potential contradictions
  • supporting evidence
  • relationships requiring further review
  • unrelated evidence

Each AI flag includes both source quotations, document references, page numbers, and the model's reasoning.

A reviewer can jump directly from a flagged relationship to the corresponding PDF page and highlighted source passage.

EvidenceWeave also provides a statement timeline to show how a person's account changes across different documents.

Human review is stored separately from model output. Lawyers can confirm, reject, or mark a relationship as uncertain and add their own notes. AI output never overwrites the human decision.

How we built it

EvidenceWeave uses a fully local analysis pipeline:

PDF → text extraction → statements → atomic facts → candidate matching → local model verification → source tracing → human review

The frontend is built with React, TypeScript, Vite, Tailwind CSS, and PDF.js.

The backend uses FastAPI, Pydantic, SQLAlchemy, SQLite, and PyMuPDF.

AI inference runs locally through Ollama using Qwen3 models. The default model is Qwen3 14B, with Qwen3 8B available as a lower-resource fallback.

Source grounding is enforced throughout the pipeline. Extracted quotations must correspond to continuous text from the original document. Facts and relationships are validated against structured schemas before being accepted.

Candidate matching rules narrow the comparison space before model verification, instead of asking the model to compare every fact with every other fact.

All case data, original PDFs, model processing, review results, and application assets remain on the local machine.

OpenAI Codex was used as an AI-assisted development tool during implementation. The submitted application itself runs locally and does not require OpenAI services or API access.

Challenges we ran into

One major challenge was preventing AI-generated relationships from becoming detached from their original evidence.

A plausible explanation is not enough in a legal workflow. We therefore treated source traceability as a system requirement rather than an optional feature.

Another challenge was distinguishing genuinely comparable facts from facts that only appear linguistically similar. Comparing every extracted statement with every other statement creates excessive noise, so EvidenceWeave first narrows candidates using subjects, event types, time information, and comparable objects.

We also had to keep model output and attorney review separate. A lawyer's review must remain an independent record and should never be silently replaced by a later model result.

Finally, local inference introduced practical constraints such as model availability, memory usage, processing time, and document quality. The system reports these conditions instead of silently switching to remote services.

Accomplishments that we're proud of

We built an end-to-end working local workflow that can:

  • import and preserve original PDF evidence
  • extract structured facts from documents
  • identify potentially conflicting or supporting evidence
  • trace every relationship back to exact source passages
  • display evidence through a consistency matrix
  • track changes across repeated statements
  • preserve independent attorney review
  • run without sending case materials to a cloud model

We also created a synthetic benchmark containing 17 PDF documents and manually labeled evidence relationships.

On the current Qwen3 14B benchmark run, the system imported all 17 documents, extracted 34 facts, and generated 94 relationships.

For the fixed benchmark relationships, contradiction precision was 90.91%, contradiction recall was 100%, source-location accuracy was 100%, and overall accuracy across the 20 labeled relationships was 95%.

These results are based on a small synthetic benchmark and are not intended as claims about performance on real criminal cases.

What we learned

The project reinforced that useful legal AI does not need to replace professional judgment.

For evidence review, provenance, uncertainty, and reviewability can matter as much as the model's ability to generate an answer.

We also learned that a smaller, constrained pipeline can be more useful than an unrestricted legal chatbot. Restricting the model to evidence extraction and relationship analysis made its output easier to inspect, verify, and challenge.

What's next for EvidenceWeave

The current version focuses on local criminal-evidence cross-checking.

Future work could include stronger OCR support for scanned case files, improved extraction for complex event sequences and indirect statements, larger benchmark datasets, and better handling of long or highly fragmented case materials.

We would also like to evaluate EvidenceWeave on more realistic anonymized case materials and study how lawyers interact with AI-generated evidence relationships during actual review workflows.

The design principle will remain the same:

AI flags. Lawyers decide.

Built With

Share this project:

Updates

Submission history