Decoder

A confidence-calibrated subtext reader for neurodiverse teens - it shows what a message might mean and, honestly, how sure it is.

Comment

Inspiration

I have a friend who reads every text three or four times before replying. Not because she can't read - because she can't tell how it was meant. "k." Is that fine, or is she mad? "wow great job 🙃" - joke, or dig? For a lot of neurodiverse people, that gap between the words and the tone is exhausting, and it's where a whole day's anxiety can live.

So I looked at what already exists. And what I found honestly scared me off the obvious idea. In May 2026, Common Sense Media and Stanford's Brainstorm Lab tested AI "support" chatbots with teens and found them unacceptably risky - they miss warning signs and keep confidently giving advice when they should stop. The lesson I took wasn't "don't build for teens." It was: the danger isn't AI being wrong. It's AI being wrong while sounding sure.

That reframed the entire project. I didn't want to build a thing that tells a kid what a message means. I wanted to build a thing that's honest about how much it actually knows - and teaches them to read tone themselves, so they need it less over time.

What it does

You paste a confusing message (or a screenshot), and Decoder shows you two or three things it might mean - each with a confidence number and the exact cue it rests on ("rests on the 🙃"). Then it offers a couple of replies, and an always-there reminder that it's okay to just ask the person.

The part I care about most: that confidence number is real. When Decoder says "56%," it means 9 out of 16 independent reads landed on that meaning - not a number the AI made up about itself. (On clear-cut messages it may stop after 6 reads; on ambiguous ones it uses the full budget up to 16.) When the reads genuinely disagree, Decoder says "this one's truly unclear" instead of forcing a guess - or shows low-confidence bars so doubt stays visible. If a message looks like it's about something heavier than tone, it stops reading tone entirely and points to a real person.

It keeps nothing. No account, no saved messages on the server. The stuff that makes it feel like yours - how you text, the cues you're learning, your history - lives only in your device.

Try it: https://decoder-dun.vercel.app/app

How we built it

I built it in three layers, and I refused to let the next layer start until the one under it was proven.

Layer 1 - the honest core. The hard insight from the research is that you cannot ask a language model "how confident are you?" - its self-reported confidence is badly miscalibrated (the literature puts the error around 0.30, basically "sounds 90% sure, is right 60% of the time"). So Decoder never asks. It runs up to 16 independent interpretations, clusters the results by meaning, and defines confidence as how many reads agree - then caps display at 90% because subtext should never look certain. Every reading's cue is verified against the actual text in code - if the model cites a cue that isn't there, that reading is dropped. AI proposes, code disposes. I also built isotonic calibration (scikit-learn) and ECE measurement in the test/eval layer; production shows raw agreement because it's already grounded in counts.

Layer 2 - the real product. A FastAPI backend + React (Vite) frontend, deployed split on Railway (API) and Vercel (UI). Screenshot parsing via AI vision, with graceful "I couldn't read this - paste the text?" fallbacks. The signature moment: when you hit Read it, you watch particles scatter in doubt and then physically converge into the confidence bars. The calibration method, made visible.

Layer 3 - safe to deploy. A tiered safety layer that catches not just explicit crisis language but the indirect register chatbots miss ("what's even the point anymore"). An abuse guard so it can't be turned into a "decode my ex" surveillance toy. EXIF stripping, hashed in-memory response cache (no readable logs), per-IP rate limits, and a privacy posture designed for teens (no accounts, on-device personalization, crisis routing to regional helplines). A calibration-drift monitor module is implemented and tested - wiring it to scheduled CI is next.

AI is used only for interpretation, vision parse, and reply coaching - never as the thing that decides the confidence numbers.

Challenges we ran into

The first real gut-check came when my own evaluation looked too good. My early calibration error came back suspiciously low on a tiny sample. Instead of celebrating, I dug in, added a confidence ceiling (the product should never show "100% sure" about subtext), and re-measured honestly. Choosing the uglier, true number over the pretty, fake one ended up being the most important decision in the project.

The second was the sample budget. At 8 reads, Decoder couldn't tell a 70/30 message from a coin flip and stayed silent on too many messages. Bumping to 16 (with adaptive early-stop when reads already agree) fixed that tradeoff between "how sure can we be" and "how often can we even answer."

And the animation fought me. My first convergence played on a separate canvas, then cut to the bars - two disconnected beats. Getting the particles to fly into the actual on-screen position of each bar and dissolve into it, as one motion, took rebuilding the whole render flow. It was worth it: that's the moment that makes people get it.

Accomplishments that we're proud of

  • A confidence number that actually means something - sample agreement you can audit, not a model's self-guess.
  • An animation that turns a statistics concept into something you feel.
  • Choosing honesty over a prettier demo, more than once.
  • A product for a vulnerable group that's safe by architecture, not by disclaimer - it keeps nothing server-side, refuses misuse, and routes crises to humans.
  • 45 passing tests across seven suites, including red-team safety tests and parser/intake coverage.
  • Shipped and live: Vercel + Railway split deploy.

What we learned

That the hard part of building with AI right now isn't getting it to answer - it's getting it to know, and show, when it doesn't. Calibration is a feature. Doubt, shown honestly, is a feature. And for the people I built this for, it might be the only feature that matters.

I also learned to distrust my own good results. The best thing I did all build was stare at a number that looked too good and ask "why is this too good?"

What's next for Decoder

Two things that need humans, not code: a lawyer to sign off on the privacy policy I've drafted, and a small validation study to prove Decoder reduces the re-reading spiral rather than feeding it (the annotation aggregation instrument is built in code).

After that: fit and deploy the isotonic calibration map on a real multi-annotator corpus (with neurodiverse annotators), wire the drift monitor to CI, deeper chat-app screenshot coverage, and native share-sheet integration from other apps.

Built With

Share this project:

Updates