ScamLearn

ScamLearn helps turn scam awareness into practice. ScamLearn will analyze suspicious activity and teach them how to think when they're under pressure

Comment

ScamLearn

Don't just detect scams. Learn how scammers manipulate you.

ScamLearn is an interactive cybersecurity training platform that uses AI to teach users how social engineering attacks work by letting them experience them.

Inspiration

As someone who is planning to pursue a career in cybersecurity, I have always wanted to get involved in projects that allow me to give back to the community. This became especially personal because my parents and brother are not particularly tech-savvy, and I sometimes worry that I may not always be able to help them when they encounter scams or other cybersecurity threats.

This can even happen to me. My messaging app once had a period where I would receive scam messages almost every day. Scammers from all over the world would claim that my Ninja Van or Amazon delivery had been delayed, sent to the wrong location, or required additional verification. They would then ask me to click on a suspicious link to "check my order."

That made me think:

What if instead of simply telling people that something is a scam, we could actually teach them how scammers manipulate their victims?

That became the idea behind ScamLearn.

What it does

Instead of simply providing a warning such as "Scam" or "Safe," ScamLearn puts users inside realistic scenarios where Claude roleplays as a fictional character attempting to manipulate them.

During a scenario, users can:

  • Question the request
  • Verify information independently
  • Refuse to cooperate
  • Provide information
  • Fall for the manipulation

The goal is not simply to tell users whether they made the "correct" decision. ScamLearn shows them how the attacker tried to manipulate their decision-making and where they could have stopped the attack.

Analyze suspicious messages

We also wanted ScamLearn to work with situations that people actually encounter in their daily lives.

Users can:

  • Paste a suspicious message
  • Upload a screenshot

ScamLearn analyzes these artifacts for evidence such as:

  • Urgency
  • Authority impersonation
  • Unusual instructions
  • Credential requests
  • Financial manipulation
  • Suspicious links or other red flags

Rather than simply returning "This is a scam," ScamLearn explains the evidence and the manipulation techniques behind it.

Users can then turn that analysis into a fictionalized training scenario.

The ScamLearn Learning Loop

Analyze → Understand → Simulate → Decide → Replay → Improve

How we built it

The core training experience is built around a deterministic scenario engine, with Claude providing the dynamic roleplay.

Training Workflow

User
 ↓
Scenario Engine
 ↓
State + Scoring
 ↓
Claude AI Roleplay
 ↓
User Decision
 ↓
Scenario Engine
 ↓
Outcome + Skill Profile

A key design decision was to keep the security logic on the backend rather than allowing the AI to determine scores or outcomes.

The scenario engine controls:

  • Scenario state
  • Scoring
  • Terminal conditions
  • Final classification
  • Skill-profile updates

Claude is responsible for dynamically roleplaying the fictional character.

Turning Real-World Scams Into Training

We also built an artifact-analysis pipeline that can turn suspicious messages and screenshots into training scenarios.

Screenshot / Message
        ↓
   AI Analysis
        ↓
Evidence + Techniques
        ↓
Scenario Generation
        ↓
Deterministic Scenario Engine
        ↓
AI Roleplay
        ↓
     Training

Generated scenarios use fictionalized entities and go through the same validation and deterministic engine as our manually created scenarios.

AI Development

Since I built ScamLearn solo, I primarily used Claude for debugging, implementation assistance, dynamic roleplay, and narrative generation, while using GPT extensively for brainstorming and refining the project's concept and direction.

Challenges we ran into

One of the biggest challenges was making the AI dynamic without giving it too much control.

I wanted the scenarios to feel like genuine conversations rather than a simple decision tree, but Claude still needed to remain within the boundaries of each scenario. It had to convincingly play the role of a fictional scammer without making the scenario unrealistically easy or impossibly difficult.

This led to an important architectural decision:

Claude controls the conversation, but not the underlying game state.

The deterministic scenario engine decides what happens after each user decision, while Claude determines how the fictional character communicates that outcome.

Another challenge was ensuring that user-controlled inputs, such as screenshots and messages, could not compromise the application. This required treating uploaded artifacts and AI-generated content as untrusted data and validating them before allowing them into the scenario engine.

Accomplishments that we're proud of

One of the things I am most proud of is completing my first hackathon entirely solo.

Having the freedom to build something related to cybersecurity, which is an area I am genuinely interested in pursuing, made the experience especially meaningful to me.

From a technical perspective, I am particularly proud of the security architecture behind ScamLearn. I wanted to make sure that users could not simply manipulate their own scores or reveal hidden scenario information, and that AI-generated content could not freely control the application's underlying logic.

I also designed the artifact-analysis system so that user-submitted content is treated as untrusted input.

In particular, I deliberately did not implement arbitrary server-side URL fetching in the current version because doing so safely would introduce additional SSRF risks.

ScamLearn currently has 100+ automated tests, alongside live testing with the real Claude API.

What we learned

One of the biggest lessons I learned was that AI output cannot simply be trusted, even when you explicitly ask for structured data.

During live testing, Claude returned an array as a comma-separated string even though the expected tool schema specified that the field should be an array. My mocked tests did not catch this because the mock was behaving exactly as expected.

Testing against the real API exposed the issue.

This taught me that AI should be treated as a semi-untrusted component of the system. Its outputs need to be validated just like any other untrusted input.

I also learned that AI and deterministic systems can complement each other well.

Claude provides the flexibility needed for realistic conversations, while deterministic backend logic provides consistency, security, and predictable outcomes.

What's next for ScamLearn

There are several directions I would like to explore in the future:

  • Safer URL analysis with appropriate SSRF protections
  • Mobile integration
  • More sophisticated behavioral analysis
  • A larger library of real-world social engineering patterns
  • More personalized training based on individual weaknesses
  • Incentives or rewards for users who demonstrate strong scam resistance

Ultimately, I would like ScamLearn to become more than just a scam-awareness tool.

I want it to help people practice cybersecurity decision-making in a safe environment before they have to make those decisions in the real world.

Built With

Share this project:

Updates

Submission history