Inspiration
We were building a normal market terminal. Then one afternoon we ran it with the WiFi off, mostly by accident, and it kept working.
Prices showed up. Fundamentals showed up. A market cap of exactly $250 billion, a P/E of 24.5, and a full table of top gainers and losers. All of it came from our own fallback code, in the same font and the same color as the real thing, with nothing on screen to tell you the difference.
One function was calculating a stock's daily percent change from the letters in its ticker symbol. It added up the character codes and did some arithmetic on them. That was the number a user would have read as a real price move.
Nobody wrote that to be dishonest. It happened the way it always happens, one reasonable-looking except block at a time, each one just trying to keep the page from crashing.
But it made the real problem obvious. People are already making money decisions on AI answers they cannot check. The issue is not that models are dumb. It is that when you read an answer, nothing tells you which parts came from real data and which the model made up. Every number looks equally confident.
We had built a small version of that exact problem without noticing, so we decided to go build the fix instead.
What it does
MarketLens is a market terminal where AI research runs as a workflow instead of a chat.
You pick what you want to know, and it runs a pipeline of real tools over real market data while you watch. Prices, indicators, fundamentals, news, and sentiment.
Then it writes you a memo where every single number is a receipt.
Hover over any figure and you can see:
- The exact tool call that measured it
- Which provider it came from
- How long ago it was retrieved
The part we care most about is what happens when the model gets something wrong.
We do not just ask it to cite sources, because a model can happily stick a convincing-looking citation on a number it invented. Instead, MarketLens reads its own finished memo back, extracts every number in it, and checks each one against what the tools actually returned.
There are four possible outcomes for any figure:
[Insert your four verification outcomes here]
Anything that fails shows up flagged directly in the memo, in front of the reader, rather than being quietly fixed or hidden.
And if a data source goes down, the run stops and no memo gets written at all.
Even with a perfectly good model sitting there waiting.
MarketLens would rather tell you it does not know.
How we built it
Everything follows from one decision:
The workflow picks the tools, not the model.
Each workflow is a graph of steps. The engine runs them in dependency order and fires independent ones in parallel, so fetching prices, fundamentals, and news can all happen at once.
The language model never chooses what to call. Its only job is to reason over data it did not select.
We gave up some flexibility there and got something better in return. The same workflow on the same day makes the same calls, so a run can be replayed and audited.
It also cuts out one of the flakiest parts of most agent systems: the model wandering off and calling something strange.
Underneath sits the fact ledger.
Every number a tool produces gets registered with:
- An ID
- A value
- A unit
- Its source
That table goes into the prompt, and the model is told it can only cite facts from the ledger.
Afterward, we compare each number in the memo against the fact it cited, allowing for honest rounding but nothing more.
Writing 182.40 for a measured value of 182.3956 is fine.
Writing 192.40 is not.
Then we had to go back and fix the data layer, because a ledger built on invented numbers would be worthless.
Every fetcher now either returns something it actually measured or raises an error. Nothing returns a plausible substitute anymore.
On top of that, we built a design system where every color, size, and spacing value comes from tokens. The entire terminal can reskin across four palettes and three densities, with contrast guaranteed.
We also built a sign-in security layer with trained models for spotting impossible travel between logins and detecting bots.
Challenges we ran into
Our first model scored 100 percent, and that was the bug
The login risk classifier came back with a perfect AUC of 1.0000.
That is not necessarily a win. It is a warning.
It meant the fake attackers and fake legitimate users we generated were trivially different, and the model had found some giveaway instead of learning anything useful.
We had made every attacker noisy and every legitimate user clean.
Fixing it meant adding the cases that actually trip real systems up:
- Someone fumbling their password four times
- A laptop bought while traveling
- An attacker renting a residential IP
- Malware replaying a stolen device fingerprint
The model landed at an AUC of 0.966, which is a number we actually trust.
A real model found a bug that our fake ones never could
We tested the verifier for ages against stub models that wrote exactly the citations we expected.
The first time we pointed a real 70B model at it, the model wrote:
its SMA-20 of $314.20
Our verifier flagged the 20 as an unverified claim.
It had interpreted a parameter name as a claim about the market.
Three angry red warnings appeared on a memo that was completely correct.
That taught us something uncomfortable:
Stubs only ever confirm what you already assumed.
Merging two branches that had both rewritten the same 3,000-line file
Eighteen commits each way.
What caught us out is that when you resolve a conflict in favor of one side, you can drop the other side's definitions while code that uses those definitions in a quiet part of the file survives untouched.
Git merges those lines without a word.
Four broken references made it through and only blew up at runtime, one crash at a time.
Running a linter over the merged result found all of them in a single pass.
That is a lesson we will not forget.
Slow code hiding in plain sight
The RSI chart was recalculating the entire history for every single bar on the chart.
Fixing it meant reproducing the original smoothing formula exactly so no chart in the app would silently change shape.
It went from 51 ms to 0.4 ms, and we tested the optimized version against the old version bit for bit to prove nothing moved.
Accomplishments that we're proud of
The moment where it refuses to answer
Type in a symbol that does not exist and watch the pipeline go red.
The dependent steps get skipped.
The memo simply never appears.
There is a working language model sitting right there, unused.
Every demo we have shown, that is the part people remember.
We caught a real model lying, live
Not in a test with a rigged stub.
We wrote a prompt deliberately telling the model to state a forward P/E, an analyst price target, revenue growth, and a dividend yield, with exact numbers.
It refused four times and said the data was not available.
When we forced a bad number through anyway, the verifier flagged it as a mismatch and showed the reader.
Zero invented numbers left
We went through and tore out every fabrication we had:
- The fake company
- The fake market movers
- The fake insider trades attributed to real named executives
- The machine learning model reporting a backtest accuracy it had never computed
You can grep the codebase for the tricks we used and find nothing.
158 tests, all offline, running in about a second
This project had zero tests when we started.
The most valuable ones are the adversarial cases, where we hand the verifier a memo that cites a real fact but states the wrong number and make sure it gets caught every time.
We found a bug that had been silently broken forever
News fetching had been failing on every single request because of a certificate problem, and a bare except was turning that failure into an empty list.
The app could not tell the difference between:
"There is no news today."
and
"Our networking is broken."
So neither could anyone using it.
What we learned
A merge that compiles is not necessarily a merge that works.
A perfect score can mean you tested the wrong thing.
Stub data agrees with you and real systems argue back, which is exactly why you need to plug the real thing in early.
But mostly, we learned that honesty is something you build, not something you claim.
Our verifier is only believable because we first went and ripped out every place the app was making things up.
When someone asks how they know the data underneath is not lying too, we can point them at the commits instead of asking them to trust us.
What's next for MarketLens
We want to add run history, so you can ask the same question a week later and see exactly which facts moved and which stayed put.
We want saved portfolios that survive a refresh.
And we want to push citation checking out of the workflow tab and into every corner of the app, so that no number anywhere appears without something real standing behind it.
Log in or sign up for Devpost to join the conversation.