Automated resume screening

Every score backed by proof — resume screening you can verify.

Comment

Inspiration

Most resume screening systems — even AI-powered ones — give scores with no explanation. HR receives a number but has no way to verify why. Worse, traditional ATS tools reject qualified candidates over simple keyword mismatches (e.g., "B.Tech CSE" rejected for "Bachelor's in Computer Science"). We wanted a system where every score comes with proof that HR can click and verify against the actual resume.

What it does

The system takes a job description and resumes as input, then runs a 6-node pipeline:

  1. ResuShield scans for hidden text fraud and prompt injection by comparing raw PDF data against OCR output
  2. Criteria Generation uses a 70B LLM to extract key requirements from the job description
  3. RAG Evaluation chunks each resume into 100-word segments, retrieves the top-3 relevant chunks per criterion, and scores them — the LLM must cite exact resume text as proof
  4. Citation Validation verifies every quote using 4 strategies: exact match, fuzzy substring, keyword overlap, and semantic similarity — failed citations auto-reduce scores
  5. Summary Generation writes professional summaries using only verified evidence
  6. Ranking sorts candidates by weighted scores — pure math, no AI

The UI has 5 tabs: Rankings with clickable citations, Validation Log, Comparison Mode (before vs after validation), Pipeline Log, and How It Works.

The final score is computed as:

$$\text{Final Score} = \frac{\sum_{i=1}^{n} s_i \times w_i}{\sum_{i=1}^{n} 5 \times w_i} \times 100$$

where $s_i$ is the validated score and $w_i$ is the weight (5 for critical criteria, 3 for others).

How we built it

  • Python + Streamlit for the web interface
  • Groq Cloud API (llama-3.3-70b-versatile) for all LLM calls with a custom thread-safe API key rotator
  • Sentence Transformers (all-MiniLM-L6-v2) for RAG retrieval and citation validation
  • PyMuPDF for PDF text extraction and bounding box highlighting
  • Tesseract OCR for extracting only human-readable text in the security module

The architecture deliberately splits AI from deterministic logic — 3 nodes use LLMs, 3 nodes are purely rule-based. This makes half the system fully reproducible and verifiable.

Challenges we ran into

  • LLM hallucinations — Started with a local 3B model that fabricated citations constantly. Migrating to a 70B model via Groq cloud fixed most issues, but the citation validator still catches edge cases — proving the validation layer is necessary regardless of model size.

  • Rate limiting — Groq's free tier has tight limits. Built a thread-safe API key rotator that cycles through multiple keys and automatically cools down rate-limited ones with per-key cooldowns.

  • Citation threshold tuning — Too high (0.80) rejected valid paraphrased citations. Too low (0.40) accepted fabricated ones. Settled on 0.55 with a multi-strategy approach that tries an exact match first before falling back to semantic similarity.

  • Hidden text attacks — Discovered that some resumes hide white-on-white keywords to game ATS systems. Built ResuShield to compare raw PDF text against OCR output to catch this.

Accomplishments that we're proud of

  • Every score has clickable proof — click any citation in the UI, and it opens the PDF page with the exact text highlighted in green
  • Anti-hallucination actually works — the Comparison Mode tab shows real before vs after scores, proving the system catches fabricated claims
  • Screened 10 resumes in under 7 minutes with zero manual intervention on a single laptop
  • 50% deterministic pipeline — half the system produces identical results every run, making it auditable and reproducible
  • ResuShield — a security layer that catches invisible keyword stuffing and prompt injection before any AI evaluation happens

What we learned

  • LLMs need guardrails, not trust. Even 70B models hallucinate. The citation validator exists because no model is reliable enough to skip verification.
  • Small models can't follow complex instructions. Citation-based prompts, JSON output, and multi-step reasoning require larger models.
  • Semantic equivalence must be explicitly instructed. Without prompt-level rules, the LLM rejects "B.Tech CSE" as not matching "Bachelor's in Computer Science."
  • Deterministic beats LLM for structured tasks. Ranking, validation, and score calculation all work better as code than as prompts.
  • Targeted fixes beat full rewrites. Iterative debugging with diagnostic scripts saved more time than rewriting entire modules.

What's next for Automated Resume Screening

  • Larger evaluation benchmarks — test with 50–200 resumes across multiple job descriptions to produce formal precision/recall metrics for the citation validator
  • Upgraded embedding models — swap all-MiniLM-L6-v2 for larger models like bge-large-en or e5-large for better semantic matching accuracy
  • Fairness and bias analysis — measure whether the system scores differently across demographics and add bias mitigation if needed
  • Multi-language support — extend ResuShield and the evaluation pipeline to handle resumes in languages other than English
  • Publishable research — the citation-grounded evaluation approach and prompt injection detection in resumes are both viable topics for an IEEE or similar paper

Built With

Share this project:

Updates