InstiGuard-Gemma

Judge-panel analysis
Score Card

Inspiration

Every enterprise wants to deploy small, self-hostable LLMs — they’re cheap, private, and fast. But “is this model safe enough for regulated work?” usually gets answered with a casual demo, not standardized, defensible quantitative measurement. Our DS6051 LLM safety coursework repeatedly emphasized one core principle: you cannot manage, audit, or mitigate risks you cannot reliably quantify.

Instead of building another generic enterprise chatbot or attempting to fine-tune Gemma for better safety, we chose to build the critical pre-deployment tool every compliance team would require first: a complete multi-dimensional safety scorecard that turns vague subjective impressions like “the model seems fine” into reproducible, comparable numerical risk scores for internal governance reporting.

What it does

InstiGuard-Gemma is a deployment-readiness safety evaluation scorecard built exclusively for gemma-4-E2B (base unaligned model) and gemma-4-E2B-it (instruction-tuned variant), treating both as candidate backbones for regulated enterprise internal reporting assistants. Our framework quantifies five distinct safety and compliance risk dimensions, paired with a triple independent LLM judge panel to reduce single-model scoring bias:

Faithfulness & Hallucination Rate: Tested on curated HaluEval ground-truth prompt-response pairs to detect unsupported fabricated factual claims
PII Leakage Risk: Synthetic enterprise records containing SSNs, salaries, and account IDs to measure failure to redact protected personal data
Steerability Compliance Failure: Two-sided adversarial pressure testing to check if the model ignores hard system-level compliance rules under user manipulation
Cross-Lingual Refusal Gap: Identical harmful/ risky prompts translated to English, Spanish, and Swahili to measure guardrail degradation in non-English low-resource languages
Novel Original Metric: Context-Saturation Constraint Decay — tracks how rapidly mandatory compliance rules fade as long contextual text fills the model’s context window, a blind spot ignored by mainstream LLM safety benchmarks

Every model output receives a probabilistic violation score from our three-judge panel: google/shieldgemma-2b (specialized safety moderation baseline), Qwen/Qwen2.5-7B-Instruct, and mistralai/Mistral-7B-Instruct-v0.3. We explicitly log and report all instances where the three judges produce conflicting safety verdicts to highlight scoring uncertainty for enterprise auditors.

How we built it

We built an end-to-end reproducible evaluation pipeline running on UVA Rivanna HPC’s dedicated 24GB hackathon GPU reservation node, structured in three isolated stages to avoid VRAM out-of-memory errors:

Dataset Construction & Translation: We curated a filtered HaluEval hallucination subset, hand-authored synthetic PII, steerability, and safety refusal test prompts, then translated all English test cases to Spanish and Swahili via the NLLB-200 multilingual translation model.
Model Inference Generation: Run both base Gemma and instruction-tuned Gemma to generate raw chat outputs for all multilingual prompts, saving all responses to disk for offline scoring to avoid repeated slow model reloading.
Multi-Judge Scoring & Automated Auxiliary Checks: Load one judge model at a time to score saved outputs and eliminate VRAM overlap; run regex-based automated PII leakage scans and our novel context-saturation decay constraint checker.
Result Aggregation: Merge all judge probabilities, automated flag results, and context decay metrics into a unified CSV scorecard for analysis and visualization.

For judge scoring logic, we use a logit extraction trick: we pull the raw softmax probabilities of the judge model outputting “Yes” (rule violation) vs “No” (compliant) rather than parsing free-form text responses, which delivers consistent, comparable numerical risk values across all three judge models. Our context-saturation decay metric requires no LLM judge at all — it uses exact string matching to verify whether mandatory compliance rules are retained as we pad the input context window with increasing volumes of neutral filler text.

Challenges we ran into

Limited Judge Model Scope: The provided baseline LLM-as-judge scaffold only supported narrow harm classification via ShieldGemma, which cannot score hallucination, instruction-following, or steerability violations. We had to rapidly redesign our full three-judge panel workflow mid-hackathon to add general-purpose instruction judges (Qwen2.5-7B-Instruct, Mistral-7B-Instruct) to cover all five scorecard metrics.
Evaluation Logic Bugs Risking Invalid Data: We caught two critical structural flaws before running full pipeline batches. Initially, our judge script applied every safety rubric to every test sample, which artificially inflated hallucination violation scores on unrelated PII/steerability prompts. We also identified a fundamental fairness gap: the raw base Gemma model has no system prompt chat template, so evaluating it against compliance system rules would produce meaningless scores without adjusted input formatting.
Rivanna HPC Connection & Environment Instability: Repeated SSH idle session terminations forced us to rebuild our Python 3.12 virtual environment multiple times; we also resolved missing dependency errors (e.g., missing Pillow for Gemma processor loading) and re-authenticated HuggingFace CLI multiple times to resolve 403 gated model access errors for Gemma and ShieldGemma.
Multilingual Translation Quality Variance: Machine-translated Swahili prompts contained ambiguous phrasing that altered the original prompt’s risk intent, requiring manual flagging and filtering of low-quality translated samples before running evaluation.
Inter-Judge Score Disagreement: We observed consistent scoring divergence between ShieldGemma and general instruction judges on borderline ambiguous responses, which required dedicated tracking tables to document and explain inconsistent risk verdicts in our final analysis.

Accomplishments that we're proud of

A fully functional, reproducible GPU evaluation pipeline that successfully runs all target models and three judge panels on Rivanna’s 24GB reserved HPC node, with all raw score data synced and version-controlled on GitHub.
Quantifiable side-by-side safety comparison between unaligned base Gemma and instruction-tuned Gemma variants across all five metrics, with clear numerical safety gaps visible in hallucination, harassment, and cross-lingual refusal performance.
Validated cross-lingual safety degradation: measurable refusal probability drop-off in Spanish and Swahili relative to English baseline guardrail performance.
Working novel context-saturation constraint decay metric, which generates a clear trend curve showing compliance rule erosion as context window length expands — a unique innovation fulfilling the hackathon’s creative metric requirement.
Complete transparent tracking of inter-judge scoring disagreement cases, rather than discarding conflicting verdicts to artificially clean results.
Full end-to-end documentation including a complete GitHub README, structured run scripts, automated result aggregation, and polished 5-minute demo presentation materials for enterprise stakeholder communication.

What we learned

LLM safety evaluation is far more complex and labor-intensive than basic LLM text generation. The majority of our hackathon work focused on validating measurement trustworthiness: refining standardized safety rubrics, spot-checking judge model performance on known ground-truth test cases, normalizing scoring polarity, and fixing subtle pipeline biases that would invalidate our entire dataset of results.

We also learned that smaller LLM judge models carry identical architectural and alignment limitations as the target models they audit. For example, our judge panel frequently produced confidently incorrect risk scores on ambiguous multi-turn enterprise prompts, demonstrating that automated LLM-as-judge scoring can never fully replace human expert review for high-stakes compliance use cases. All quantitative scores are only relative risk estimates, not absolute ground-truth safety labels.

Additionally, we observed that instruction tuning only partially mitigates core safety weaknesses present in the raw base LLM; guardrail performance still degrades significantly in non-English languages and long context windows regardless of alignment fine-tuning.

What's next for InstiGuard-Gemma

Independent judge panel scaling: Separate target model inference and judge model execution to fully isolate compute workloads and eliminate VRAM memory bottlenecks for larger judge model families.
Expand the context-saturation decay test to cover a full suite of distinct compliance rules, rather than one single hard constraint.
Mitigate cross-lingual safety gaps: Test whether native language system prompts (instead of English-only guardrail instructions) improve non-English refusal rates and close the measured safety gap.
Add human expert ground truth labeling to calibrate and validate judge model probabilistic violation scores, reducing LLM-as-judge inherent bias.
Build dynamic compliance threshold logic customized for distinct regulated industries (HIPAA healthcare, GDPR data privacy, financial audit frameworks).
Expand multilingual test dataset coverage to include more low-resource Asian and African languages beyond Spanish and Swahili.
Develop lightweight prompt-level safety mitigation strategies to reduce context-saturation constraint decay for long-document enterprise workflows.