Inspiration
A convincing backtest can tell the wrong story.
A strategy may appear successful because it uses information that was unavailable when a trade occurred, ignores trading costs, or relies on invalid data. A polished chart makes these mistakes easy to overlook—especially for students learning quantitative research.
We built QuantForge around a question: Can you inspect the evidence behind a backtest, challenge its assumptions, and reproduce its correction?
Our goal is to help student and independent researchers understand why a result may be unreliable before treating it as useful evidence.
What it does
QuantForge provides a complete research workflow:
- Select a supported strategy template and edit its assumptions.
- Review the exact signal, sample, execution timing, allocation, and costs.
- Approve and run a deterministic backtest.
- Inspect the computed NAV, returns, drawdown, and trade ledger.
- Examine audit findings with source rows, fields, and timestamps.
- Approve a correction that creates a separate experiment.
- Compare both runs and download the saved reports.
The current MVP checks information timing, modeled trading costs, and dataset integrity. Checks for train/test leakage, overfitting risk, and survivorship bias remain marked Unverified until the required evidence and audit capabilities are available.
A guided, intentionally flawed example uses tomorrow's closing information before an opening trade. The auditor exposes the timing violation, and the corrected run shows which entry decisions actually changed.
All included price data is clearly labeled synthetic. QuantForge does not place trades or promise investment returns.
How we built it
We built the backend with Python, FastAPI, and Pydantic. Strict schemas constrain the supported strategy and reject invalid inputs.
The numerical engine uses Decimal arithmetic to calculate trades, modeled costs, portfolio NAV, and performance metrics. Decisions record their information dependencies so the auditor can compare when an input became available against when a decision and simulated fill occurred.
SQLite stores strategy revisions, exact approvals, experiment events, and hash-checked scientific artifacts. Corrections create child experiments linked to their originals; they never overwrite the original result.
The frontend uses HTML, CSS, and JavaScript modules, served by the same backend. It renders engine-generated results, audit evidence, comparison views, history, and downloads.
Optional anonymous demo sessions provide isolated workspaces with expiry and persistent quotas. We also implemented an optional Claude planning adapter that proposes constrained parameters for review. Live-provider validation remains pending; the template workflow works independently.
We verified the current version with 146 backend tests, 10 frontend tests, and a real HTTP smoke workflow. Browser regression coverage exercises the research journey. A Render deployment blueprint is included, while public deployment verification remains a separate delivery step.
Challenges we faced
Making information timing precise
Checking whether a strategy uses a future date is not enough. We needed to distinguish information availability, decision time, order submission, and execution. Recording those relationships made the findings inspectable.
Correcting experiments without rewriting history
A correction changes the research question or its assumptions. We built explicit approval, separate run identifiers, parent–child relationships, and comparison reports so users can trace what changed.
Keeping the demonstration honest
We wanted a clear demonstration without inventing performance numbers. Both the flawed and corrected examples run through the actual engine on disclosed synthetic fixtures. Their charts and metrics come from execution, not hardcoded presentation values.
Handling failure and recovery
Invalid data, stale approvals, duplicate requests, expired sessions, and exhausted demo quotas all need understandable outcomes. Testing these boundaries uncovered issues that a successful backtest alone would not reveal.
What we learned
We learned that reproducibility requires more than saving parameters. It also requires pinned data, implementation fingerprints, explicit timing semantics, recorded decisions, and preserved outputs.
We also learned that completing a calculation and establishing its validity are different things. A run can finish successfully while its methodology remains blocked or unverified.
Most importantly, useful research software should help people understand evidence—not simply produce more convincing charts.
Accomplishments we're proud of
- A working review → backtest → audit → correction → comparison workflow.
- Evidence-backed detection of an intentionally injected timing error.
- Actual recomputation of corrected experiments with preserved originals.
- Clear separation between implemented checks and unverified risks.
- Reproducible synthetic fixtures and automated verification.
- English documentation covering setup, usage, limitations, and troubleshooting.
What's next for QuantForge
Our next steps are to verify a publicly accessible deployment, observe independent users completing the workflow, and validate the optional Claude planner against the real provider.
Longer term, we want to add appropriately licensed historical data, stronger out-of-sample evaluation, and user-confirmed research memory. These additions will preserve the same principle: models may propose and explain, while deterministic tools calculate results and establish audit findings.
Team and development disclosure
QuantForge was built by Xuefeng Qin and Lucas Chu, using the team's revised QuantForge whitepaper as the design starting point.
We used OpenAI Codex for architecture, documentation, code generation and editing, test development, debugging, and verification. Third-party libraries and tools are recorded in the repository's dependency files.
The application includes project-generated synthetic data. Development assistance from Codex is separate from runtime model functionality; we do not claim that every logical agent role is powered by a language model.
Log in or sign up for Devpost to join the conversation.