AI Literacy Check

Inspiration

Two students write the same sentence: “Weekly AI users scored three points higher on the quiz.” One checked the original report. The other copied the assistant’s answer. A teacher reading the finished briefs would see the same figure. We built AI Literacy Check to give each student feedback on how they got there.

What it does

Students have four minutes to write an 80–150 word briefing for a teaching committee. Their question is simple: does using AI help first-year students learn? They share a fictional source pack with an AI research assistant and decide what to ask, which claims to check and what to include.

The three-point gap is real. The quiz report also shows that weekly AI users attended more seminars. Students have to decide how much the evidence supports saying about cause and effect.

After submission, students can expand each finding to see what prompted it and what they could try next time. There is no overall score or ranking. The eight-minute version adds up to two adaptive questions so students can explain decisions that their chat and brief left unclear.

How we built it

We built the workspace with Next.js, React and TypeScript. Node.js and SQLite record chat messages, source opens, edits and the submitted brief. Claude reviews redacted session text, and the server checks that the messages cited as evidence exist. Teachers can review disputed feedback.

Challenges and what we learned

Our first rules sometimes missed a student’s point when they expressed it in their own words. We tried several phrasings of the same idea and compared sessions where students checked an AI claim with sessions where they used it directly.

That exposed a second challenge: opening a report shows that a student saw it, but tells us little about what they understood. The most useful feedback points to a specific decision, such as keeping the quiz figures while dropping the claim that AI caused higher marks. When the reasoning is still unclear, the full version can ask the student why.