Inspiration

Construction disputes are decided by forensic schedule analysts who manually reconstruct causal chains — differing site condition → design freeze → weather delay → critical path impact — from thousands of pages of RFIs, change orders, and correspondence. Meanwhile, every "AI delay prediction" tool we found does the opposite: it trains a flat tabular model (resource scores, cost ratios) and spits out a risk percentage with zero causal reasoning behind it. Nobody had connected the two. We wanted to know: when a delay-prediction model gets it wrong, can we use real legal precedent to explain why — and eventually, to make it wrong less often?

What it does (Phase 1 — this hackathon)

CONCRETE EVIDENCE is a diagnostic pipeline that cross-checks a predictive delay-risk model's failures against real construction-dispute case law:

The predictor — an XGBoost classifier trained on a 1,301-row construction task dataset, engineered into 14 features (resource constraint, site constraint, cost-per-labor, dependency count, etc.), with full SHAP explainability on every prediction.
The forensic layer — 22 real U.S. construction delay disputes (ASBCA appeals, federal court opinions) pulled from CourtListener, LLM-scored for causal richness, then parsed into structured causal DAGs: nodes are events (differing_site_conditions, weather_delay, liquidated_damages_trigger...), edges are typed relationships (caused, contributed_to), each tagged with an evidentiary confidence tier — court_finding and explicit outrank alleged_only and contested — so a judge's finding of fact is weighted differently from a party's unproven allegation. 16 of the 22 cases yielded rich enough chains to extract.
The alignment layer — a single taxonomy.yaml crosswalks the predictive model's feature names to the forensic event types, so the two independently-built systems can actually talk to each other.
The cross-check — on every misclassified test-set task, we take the top SHAP-driving features, map them through the taxonomy to forensic event types, query the case database, rank matches, and DFS-walk the matched case's DAG from the anchor node outward. The result is a plain-English diagnosis: not just that the model was wrong, but which real legal precedent shows the causal cascade it missed.

On our test set: 260 tasks, 28 misclassifications (10.77% error rate), and 27 of those 28 (96.4%) matched to a real forensic precedent explaining the gap.

This is deliberately Phase 1: right now the forensic layer is read-only — it explains the model's blind spots after the fact. It doesn't yet fix them. That's the actual research question we're going after next.

How we built it

Fetched and LLM-scored 22 CourtListener construction-delay opinions for causal richness (fetch_courtlistener_cases.py, score_cases_nvidia_llm.py) using NVIDIA NIM.
Extracted structured causal DAGs from the scored case text (extract_dags.py), validated against a hand-built taxonomy of 9 delay categories and ~20 event types, stored in a queryable SQLite database (forensic_dags.db).
In parallel, trained a baseline XGBoost delay-risk classifier on a Kaggle construction dataset with SHAP interpretability (train_baseline.py).
Built the merge pipeline (predict_test_set.py → precedent_matcher.py → error_analysis.py) that runs the predictor, isolates its failures, and cross-references them against the forensic case DAGs to generate a structured, cited diagnosis for each error.

Challenges we ran into

Two branches, two vocabularies. The predictive model's feature names and the forensic taxonomy's event types were built independently by two people — reconciling them into one alignment layer that wouldn't silently drift as either side changed was its own design problem.
Not all evidence is equal. Legal text mixes court findings, expert opinions, and unproven allegations in the same paragraph. We built a confidence-tier system so precedent matching doesn't treat a disputed claim as fact.
LLM API instability mid-pipeline — rate limits forced a fallback path between providers partway through DAG extraction, so the extraction script had to be resilient to switching providers without corrupting partially-processed cases.

Accomplishments that we're proud of

Getting a 96.4% precedent-match rate on real misclassifications — almost every time our model got it wrong, we could point to an actual court case showing the causal mechanism it failed to represent. That validates the core thesis: flat-feature models miss multi-hop causal cascades, and real case law can quantify exactly where. It's a strong foundation to build the actual fix on.

What we learned

The diagnosis pattern kept repeating: XGBoost sees one elevated feature and predicts low risk, but the matched precedent shows that feature was just the first link in a chain the model has no structural way to represent, because tabular features are flat and disputes are sequential. That's not a modeling bug — it's a representation problem, and representation problems are fixable with the right features.

What's next for Concrete Evidence

This hackathon build is intentionally the ideation and validation phase — proving the diagnostic layer works before building the fix on top of it. Next:

Close the loop, not just diagnose it. Engineer causal-chain features directly from the DAGs (does this task's active event types appear upstream of critical_path_impact in precedent? what's the matched-precedent liquidated-damages rate?) and feed them back into the predictor — turning "we found the model's blind spot" into a measurably better model.
Make the diagnosis self-triggering, so instead of a human reading a markdown report, the system automatically flags which feature is structurally blind and proposes the new feature to close that gap.
Scale the precedent base and track a coverage-gap metric as it grows, plus validate our LLM-assigned causal-confidence tiers against manual review — rigor that matters if this ever touches real claims work.
Run it on live, unseen projects — not just a static test split, but a real intake path that takes a folder of schedules/RFIs/change orders for an active project and outputs a prediction with an evidence-backed causal explanation.
Visualize the causal chain, since the DAG walk is the most compelling artifact we have and it's currently buried in markdown.