GemmaShield

Inspiration

Modern AI systems are increasingly used in education and research, yet few learners truly understand how adversarial attacks work or how to build safe AI models. GemmaShield EDU was inspired by the need to make AI safety learning interactive, visual, and hands‑on. Our goal is to bridge the gap between AI capability and AI safety by allowing students to see how vulnerabilities emerge and how defenses respond — all in a controlled, local environment.

What it does

GemmaShield EDU is an interactive learning platform that simulates real AI security scenarios. It helps students and developers explore how prompt injection, jailbreaks, and data leaks occur — and teaches how to defend against them safely.

The system uses a multi‑agent architecture powered by Gemma 4 via Ollama, including:

• Attacker Agent — Generates adversarial prompts targeting the model’s system prompt. • Target Agent — Simulates the behavior of the deployed LLM under attack. • Defender Agent — Analyzes interactions and classifies vulnerabilities. • Judge Agent — Assigns a CVSS‑like score and provides mitigation recommendations.

Each simulation runs locally in real time, ensuring transparency, reproducibility, and privacy — perfect for classroom demonstrations and student labs.

How we built it

GemmaShield EDU was built using a full‑stack, real‑time architecture:

• Backend: FastAPI + Server‑Sent Events (SSE) for streaming agent outputs. • Frontend: React dashboard with a live simulation console and OWASP LLM Top 10 heatmap. • Inference Layer: Ollama REST API running Gemma 4 locally for all agent reasoning. • Database: SQLite + JSONL logs for full attack‑response traceability. • Evaluation Layer: Structured JSON outputs for consistent scoring and vulnerability classification.

This architecture allows learners to observe every stage of the attack‑defense cycle as it happens.

Challenges we ran into

One challenge was handling Gemma 4’s “thinking‑mode” output, which included intermediate reasoning before structured JSON responses. We solved this by switching to the Ollama REST API with streaming support and regex‑based filtering to extract clean structured outputs.

Another challenge was maintaining consistent evaluation across multiple adversarial scenarios while keeping the system lightweight and fully local — essential for classroom use.

Accomplishments that we’re proud of

We successfully built a fully local educational simulator capable of visualizing real‑world adversarial attacks on AI systems.

Key achievements include:

• A working multi‑agent pipeline (Attacker, Target, Defender, Judge). • Real‑time streaming of AI interactions with full traceability. • Integration of OWASP LLM Top 10 taxonomy for automated vulnerability classification. • A CVSS‑like scoring system for AI security evaluation. • A privacy‑preserving architecture that runs entirely offline.

What we learned

This project deepened our understanding of LLM security, adversarial machine learning, and multi‑agent system design. We learned how unpredictable LLM behavior can be — and how structured evaluation frameworks make AI systems safer and more auditable. Most importantly, we realized that AI security education is essential for building the next generation of responsible AI developers.

What’s next for GemmaShield EDU

We plan to evolve GemmaShield EDU into a full educational platform by adding:

• Guided lessons and classroom dashboards. • Gamified challenges for students to test their defense strategies. • Continuous integration support for automated testing in learning pipelines. • Expanded adversarial libraries beyond prompt injection (e.g., data poisoning, tool misuse).

Built With

• gemma4
• javascript
• node.js
• ollamma
• python
• react
• sqlite