FinTrace

Inspiration

AI is increasingly used to interpret financial information, but a confident answer is not necessarily a correct or verifiable answer. A model can repeat a percentage without recalculating it, accept a disclosure that explains only part of a numerical gap, or produce a convincing explanation that never appears in the filing.

These failures matter because analysts, investors, auditors, researchers, and students may rely on the result when making real decisions.

I built FinTrace around one principle:

Models interpret. Code calculates. Filings provide evidence. Humans decide.

My goal is not to replace financial professionals. It is to make AI-assisted financial review easier to inspect, challenge, and verify.

What it does

FinTrace compares statements from earnings calls with the corresponding filed financial data. It identifies potential inconsistencies, recomputes the relevant values using deterministic Python, retrieves supporting filing passages, and validates whether the evidence actually explains the difference.

For example, suppose management claims that revenue grew by 23%, while the filing reports current-period revenue of 124 and prior-period revenue of 110. FinTrace calculates:

$$ g = \frac{124 - 110}{110} \times 100 = 12.73\% $$

The difference between the claim and the filed calculation is:

$$ 23.00\% - 12.73\% = 10.27\text{ percentage points} $$

FinTrace then searches the filing for a specific explanation, such as currency effects, an acquisition, a divestiture, an accounting-policy change, or a segment reclassification.

A finding is marked EXPLAINED only when the proposed evidence:

appears verbatim in the supplied filing;
directly relates to the finding; and
reconciles the numerical difference.

If any check fails, FinTrace preserves the finding as UNRESOLVED for human review instead of allowing the model to guess.

Key features

Four scope-isolated specialist reviews for revenue, cash flow, related-party items, and observable language patterns
Deterministic Python calculations with visible inputs, formulas, results, gaps, and tolerances
Filing evidence retrieval across financial statements, footnotes, MD&A, policies, and segment disclosures
A second-pass LLM review that evaluates possible explanations
Exact-quote, relevance, and numerical-reconciliation validation
Fail-closed handling that preserves unsupported findings as UNRESOLVED
Human analyst review and final sign-off
Interactive evidence chain, architecture view, prompt inspector, benchmark comparison, and guided product tour
Downloadable JSON, HTML, and PDF artifacts for reproducibility

Target users and real-world impact

FinTrace is designed for:

financial analysts reviewing management claims;
auditors and researchers checking supporting disclosures;
investors who want a clearer evidence trail;
finance students learning how claims connect to filed numbers; and
AI developers building more accountable decision-support systems.

The project's broader impact is financial transparency. Instead of presenting AI output as an unquestionable answer, FinTrace shows the calculation, filing evidence, validation result, and remaining uncertainty. This can help users recognize unsupported claims and reduce overreliance on persuasive but weakly grounded AI analysis.

FinTrace is a research prototype and decision-support tool. It does not provide investment advice or replace professional judgment.

How I built it

I built FinTrace as an eight-stage workflow:

Ingest and align: Python maps claims, speakers, periods, metrics, and filing references.
Specialist review: Four batched LLM calls inspect the same cases through isolated financial scopes.
Aggregate and deduplicate: Deterministic Python combines overlapping findings, preserves provenance, and assigns stable identifiers. This stage is not an LLM call.
Code calculation: Python recomputes growth, free cash flow, margins, and direct-value comparisons using explicit tolerances.
Filing retrieval: Deterministic retrieval ranks claim-linked passages and relevant disclosures.
Second-pass review: A second LLM proposes EXPLAINED or UNRESOLVED using the calculation and retrieved filing evidence.
Deterministic validation: Python checks quotation accuracy, relevance, and numerical reconciliation.
Analyst review and sign-off: A human records the final decision, note, and timestamp.

The live workflow uses Python, structured prompt contracts, Groq's API, and llama-3.3-70b-versatile at temperature $T=0$. The presentation layer uses Next.js, React, TypeScript, and Vercel. ReportLab and pdfplumber generate and verify the report artifacts.

The Python engine in fintrace/ is the authoritative implementation. The public Next.js website presents a versioned fictional fixture so judges can explore the workflow without API credentials. Full live execution is available through the Python command-line interface.

Challenges I ran into

The hardest challenge was distinguishing a plausible explanation from a sufficient explanation.

In one benchmark case, the live single-prompt baseline accepted a disclosed USD 3 million item as the explanation for a USD 5 million gap. The disclosure was real and relevant, but it did not reconcile the complete difference:

$$ \text{USD }40\text{M} + \text{USD }3\text{M} = \text{USD }43\text{M} \neq \text{USD }45\text{M} $$

FinTrace therefore kept the finding UNRESOLVED.

Another challenge was preventing model arithmetic from becoming authoritative. Language models can calculate, but they can also make mistakes while sounding certain. I moved the important calculations into deterministic Python and exposed the formula and inputs in the interface.

Quotation validation was also difficult. A quote should not pass merely because it resembles filing language. I implemented an exact-quote and relevance gate that can override the model's proposed conclusion.

Live API execution introduced rate limits and retries. I made the provider fail loudly when credentials or calls fail. It never silently falls back to the local deterministic provider while presenting the result as a live run.

Finally, I had to present a complex pipeline without overwhelming users. I used a visual evidence chain, consistent verdict labels, progressive disclosure, a guided tour, and an architecture diagram to make the system understandable.

Accomplishments that I am proud of

I evaluated FinTrace on a controlled fictional benchmark containing 12 cases and 13 independently scored findings. The live single-prompt baseline and full workflow used the same model at temperature $T=0$.

Approach	Correct findings	Classification accuracy
Live single prompt	10/13	76.9%
Full FinTrace workflow	13/13	100.0%

The baseline received one broad instruction. The full workflow added specialist scopes, deterministic calculation, retrieval, second-pass review, quote validation, relevance checking, numerical reconciliation, and human oversight.

This benchmark is intentionally small and fictional. It demonstrates behavior on controlled failure modes; it does not prove general performance on all companies, filings, models, or market conditions. I report the recorded results without hiding cases where the baseline or individual specialist reasoning was imperfect.

I am also proud that:

every financial calculation is reproducible;
prompt contracts are versioned;
model responses are schema-validated;
specialist provenance remains visible;
unsupported explanations fail closed;
unresolved findings are preserved; and
the final judgment remains with a human.

What I learned

I learned that prompt engineering is not only about writing a better prompt. It is also about deciding:

which task belongs to the model;
which result must be controlled by deterministic code;
what evidence must be retrieved;
which outputs require validation;
when the system should refuse to decide; and
where human judgment must remain mandatory.

I also improved my understanding of financial reconciliation, structured LLM outputs, evidence retrieval, prompt isolation, benchmark design, retry handling, Next.js interfaces, Vercel deployment, and technical storytelling.

The most important lesson was that a reliable workflow does not require every individual model response to be perfect. Reliability can come from designing later stages that detect, constrain, and expose imperfect reasoning.

What's next for FinTrace

My next steps are to:

parse complete 10-Q and 10-K filings;
extract structured values from filing tables;
support longer earnings-call transcripts;
expand the financial-claim taxonomy;
improve retrieval and source-location tracking;
evaluate additional models and providers;
build a larger independently adjudicated benchmark;
persist richer token, cost, retry, and redirect diagnostics;
add analyst collaboration and review history; and
conduct evaluations with financial professionals.

FinTrace demonstrates a practical direction for accountable financial AI: do not trust an AI-generated financial explanation unless its calculation, evidence, and final decision can be independently verified.