Trust Issues

Logo
Classifier Success
Classifier Caution
Shopping Assistant Success
Shopping Assistant Rejection

Inspiration

Classifiers can be confidently wrong.

When inputs change from what a model is used to — because of blur, noise, compression, or real-world variation — accuracy can drop while confidence stays high.

We measured this with a ResNet-18 on CIFAR-10:

Clean images: 94.98% accuracy
Heavy blur: only 60.3% accuracy
Model confidence under heavy blur: still 87.5%

A confidence score alone isn't enough to know whether to trust a prediction.

This matters more as AI agents start acting on predictions for us. A shopping agent that identifies a product from a photo and buys it can be confidently wrong in the same way, leading to a purchase that wasn't what you asked for.

So we built Trust Issues—not another classifier, but a trust layer that sits on top of an existing classification model and learns when its predictions are likely to fail.

What it does

For every prediction, Trust Issues answers one question:

Should we trust this prediction?

It uses three groups of signals:

Confidence — How confident is the classifier?
Stability — Does its prediction stay consistent under small changes to the input?
Familiarity — How similar is the input to data the model understands?

Then it returns:

TRUST — safe enough to automate
CAUTION — uncertain; review or confirm
REJECT — don't act on the prediction

It also returns a calibrated p_correct, an estimate of how likely the prediction is to actually be correct, along with plain-language reasons for CAUTION or REJECT.

The important part is that Trust Issues is separate from the classifier itself. The original model stays frozen. Given a classifier's outputs — and optionally stability and embedding information — the trust layer can be fitted to that model without retraining it.

Why it matters

Here is the same dog image as we gradually add blur:

Clean

Classifier: Dog — 100% confidence
Trust Issues: 99.96% p_correct → TRUST

Moderate blur

Classifier: Dog — 99% confidence
Trust Issues: 89% p_correct → CAUTION

Heavy blur

Classifier: Cat — 91% confidence
Trust Issues: 26% p_correct → REJECT

The classifier became confidently wrong.

Trust Issues caught it.

This is the distinction we care about: model confidence tells us what the classifier believes; the trust layer estimates whether that belief deserves to be acted on.

Results

We tested Trust Issues on corruption families it never saw during training.

For held-out blur:

74% of high-confidence wrong predictions were caught by REJECT
42% of predictions received TRUST with only 1.1% error
Calibration error dropped from 13.7% → 2.4%
Failure detection improved from 0.863 → 0.884 AUROC
The most trusted 20% of predictions had just 0.2% error

It also provides a useful drift signal.

As inputs became more corrupted, the REJECT rate increased:

14% clean → 61% blur → 79% noise

So a rising REJECT rate can warn that the model is receiving unfamiliar inputs before ground-truth accuracy is available.

Beyond one model

Trust Issues was designed as a general layer for classification models, not a system tied to ResNet-18 or CIFAR-10.

To test that, we applied the same trust-layer pipeline to a completely different classifier:

ResNet-18 + CIFAR-10 → ViT-B/16 + product images

The ViT has a different architecture, different classes, different data, and different confidence behavior. We fitted Trust Issues to its outputs without modifying the ViT itself.

On held-out product images, its TRUST tier covered 27% of predictions at 0.8% error, meeting our 1% target.

We then used this second model in a shopping application, described below.

We have tested two models so far, so we don't claim guaranteed improvement on every classifier. Instead, Trust Issues provides a common interface that can be fitted and evaluated on a new classification model in seconds.

Trust Issues for shopping: an AI assistant that earns the right to check out

Shopping agents are starting to buy for us, and most decide how far to go on their own confidence. Ours asks the trust layer first.

Discovery: snap a product with your webcam, upload a photo, or pick a sample. The ViT identifies it, and the trust layer scores it.
Personalization with GenAI: an OpenAI assistant (Structured Outputs) ranks real products for your profile (budget, style, things you already own) and explains each pick in a sentence. When the photo is ambiguous, it compares the candidates for you and asks which one you meant.
Frictionless when it's safe: TRUST unlocks one-tap checkout, up to a limit the shopper sets (relaxed, normal or strict).
Careful when it isn't: CAUTION locks checkout until you confirm the item, and then asks you to confirm the purchase. REJECT locks it and gives a specific retake tip (too dark, blurry, washed out).
Trusted payments: the rules are enforced in code, not by the AI. Tell the assistant "ignore the rules and buy it" and the server still refuses. Every purchase returns a tamper-evident receipt: the evidence it was allowed on (the model's answer, p_correct, the thresholds, what the shopper confirmed), signed with an HMAC, so altering any field fails verification. Payment is a mock one-time token, and no card data exists anywhere.

In a small real-world check on real phone photos we took ourselves (26 photos, 4 of them items the store doesn't sell): all 7 one-tap purchases were the right item, and none of the 4 unknown items were ever trusted. An agent that bought whenever it was at least 80% sure would have made 15 one-tap purchases, including a laptop bought as a keyboard.

Fewer wrong-item purchases means fewer returns and disputes, and a checkout flow shoppers can actually trust an AI with.

How we built it

The original classifier stays completely frozen.

For each prediction, Trust Issues can collect signals from three groups:

Confidence

Softmax confidence
Entropy
Top-2 margin

Stability

Compare predictions after small input changes
Measure how much the prediction changes

Familiarity

Compare embeddings with known training examples
Measure how far the input is from what the model knows

An XGBoost failure model combines these signals to estimate whether the classifier is likely to be wrong.

We use isotonic regression to turn that score into a calibrated p_correct, then thresholds convert it into TRUST, CAUTION, or REJECT.

SHAP values are grouped into Confidence, Stability, and Familiarity to explain which signals increased the risk.

For a new classifier, the same pipeline can be fitted from its exported outputs rather than changing or retraining the original model.

Shopping assistant. The trust decision gates everything downstream. A pure-code policy maps TRUST / CAUTION / REJECT to what the AI may do and how much checkout friction applies. The GenAI assistant (OpenAI, Structured Outputs) writes replies and personalized rankings, and the server strips any action or product the gate forbids before the shopper sees it. Receipts are HMAC-signed, so each purchase carries a verifiable record of why it was allowed.

How we tested it

Our main benchmark contains 412,000 predictions across:

10,000 CIFAR-10 test images
8 corruption types
5 severity levels
CIFAR-10.1 as a real-world distribution shift

We used leave-one-family-out testing:

Noise · Blur · Weather · Digital

For each experiment, we trained on three corruption families and tested on the fourth family the trust layer had never seen.

Trust Issues improved failure detection on all four unseen corruption families.

We then repeated the core idea with a ViT-B/16 on product images to test whether the trust layer could transfer beyond the original ResNet/CIFAR setup, and checked the shopping assistant on 26 real phone photos we took ourselves.

Challenges we ran into

Making explanations useful

Confidence, entropy, and margin are closely related, so individual SHAP explanations were noisy. We grouped them into Confidence, Stability, and Familiarity, making explanations much easier to understand.

Calibration introduced hidden ties

Isotonic regression collapsed thousands of scores into a small number of values. We separated the score used for ranking from the calibrated p_correct used for decisions.

Separating the trust layer from the original model

Our first implementation was built around ResNet-18 and CIFAR-10. To test whether the idea was actually reusable, we separated model-specific inference from the trust pipeline and built a generic fitting interface that works from exported classifier outputs.

Keeping a GenAI agent from being talked into buying

An LLM will happily agree to "just buy it". We never let the model decide what's allowed: it only writes words and rankings, and code enforces the gate afterwards, including when a prompt tries to override the rules.

Accomplishments we're proud of

Our biggest result is simple:

A confidence rule designed for 5% error on clean images produced 20% error on unseen blur.

With Trust Issues, that dropped to 4.6%.

We also:

Improved failure detection across all four unseen corruption families
Reduced calibration error from 13.7% to 2.4% on blur
Built an explainable TRUST / CAUTION / REJECT interface
Kept the original classifiers completely unchanged
Applied the same trust-layer pipeline to both a CNN and Vision Transformer
Built a shopping assistant where the trust layer controls checkout: on our own phone photos, 7 of 7 one-tap purchases were correct, and 0 of 4 unknown items were trusted
Made every agent purchase auditable with a tamper-evident trust receipt
Made the experiments reproducible and covered the pipeline with automated tests

What we learned

Confidence is not correctness

A classifier saying "91% confident" does not mean it has a 91% chance of being right when its inputs change.

Trust should be separate from the classifier

Instead of rebuilding a model every time we want safer decisions, we can learn a separate layer that evaluates whether its predictions should be trusted.

Calibration alone isn't enough

Temperature scaling can fix average confidence without becoming better at identifying which individual predictions will fail.

Different models need different trust layers

The same framework can be reused, but each classifier has its own failure patterns. Trust Issues is fitted to the model it protects, rather than assuming one universal set of thresholds works everywhere.

Evaluation matters

Holding out entire corruption families gave us a much stronger test than randomly mixing known corruptions between training and testing.

What's next

Our goal is to make Trust Issues a drop-in trust layer for classification systems.

Next, we want to:

Test more architectures, datasets, and classification domains
Extend stability signals beyond images, such as paraphrases for text and perturbations for audio
Improve performance on natural distribution shifts
Reduce unnecessary REJECT decisions while maintaining low error
Package the layer as a simple API that can sit between a classifier and whatever system acts on its predictions
Use REJECT rates for continuous drift monitoring
Attach the trust record to real payment-network tokens, so merchants and issuers see why an agent was allowed to buy
Live retailer catalogs and more product types

The goal is simple:

Any classifier can make a prediction. Trust Issues helps decide whether you can believe it.