Calibrity

• 

  Calibrity's home screen introduces the core idea: learn when to Trust, Verify, or Reject AI answers and calibrate confidence to evidence.

• 

  Core interaction screen: the student reads an AI answer, chooses Trust, Verify, or Reject, and rates confidence from 0–100%.

• 

  Example of a reliable AI answer with an explanation, user decision, confidence level, and score breakdown.

• 

  Example of an uncertain AI answer where verification is the ideal action, with feedback explaining why caution is warranted.

• 

  Example of an unreliable AI answer showing how Calibrity rewards correct rejection and explains the hidden error.

• 

  Final calibration profile showing the user's score, overtrust, undertrust, verification behavior, and confidence patterns.

• 

• 

• 

• 

• 

Inspiration

AI answers can sound fluent, confident, and convincing even when they are wrong or uncertain.

Most AI literacy tools focus on spotting hallucinations. But in real life, another skill matters just as much: knowing when to trust an AI answer, when to verify it, and when to reject it.

Calibrity was built to train that judgment.

What it does

Calibrity presents students with seven simulated AI answers across topics such as science, history, statistics, technology, health literacy, and citations.

For every answer, the user chooses one of three actions:

• Trust
• Verify
• Reject

They also rate how confident they are from 0–100%.

Calibrity then compares the decision and confidence level with the reliability of the answer and measures patterns such as:

• Overtrust
• Undertrust
• Appropriate verification
• Overall confidence calibration

At the end, the student receives a calibration score and a short profile showing how well their confidence matched the available evidence.

How we built it

Calibrity is a React + TypeScript web application built with Vite.

For this prototype, the scenarios are pre-written and reviewed instead of being generated live by an AI model. This keeps the educational feedback predictable and prevents an unreliable model response from changing the result during a demonstration.

The scoring engine classifies each decision as ideal, cautious, or incorrect. Confidence strengthens correct decisions and increases the penalty for confidently incorrect decisions.

Everything runs locally in the browser. There is no account system, backend, database, or personal-data collection.

Challenges we ran into

The biggest challenge was preventing the scoring system from being gamed.

An early version gave users a relatively high score if they selected Verify for every answer. We redesigned the scoring system so that users must learn when trusting, verifying, and rejecting are each appropriate.

We also reviewed the scenarios and explanations carefully so that the reliability labels did not contradict the educational feedback.

Accomplishments that we're proud of

• Built and deployed a complete working prototype
• Created a confidence-aware calibration scoring system
• Added anti-gaming logic
• Tested scoring across hundreds of decision and confidence combinations
• Verified responsive mobile behavior
• Added keyboard accessibility
• Kept all interaction local to the user's browser

What we learned

AI literacy is not only about detecting false information.

A person can also be poorly calibrated by distrusting reliable information or by verifying everything regardless of the evidence.

Useful AI judgment means matching your level of confidence to the quality of the evidence.

What's next

Future versions could include:

• Adaptive scenarios based on previous mistakes
• More subject-specific scenario packs
• Classroom and teacher tools
• Source-verification exercises
• Longer-term calibration tracking
• Optional live AI-answer exercises

Built With

• css
• html
• react
• typescript
• vercel
• vite