Job Guard

JobGuard detects fake job postings using dual-branch AI — DistilBERT analyses language, XGBoost flags structural red flags. Every prediction is explained, not just scored.

Comment

JobGuard — Fake Job Posting Detector

Inspiration

Job scams are not a niche problem. The FTC reported over $220 million lost to employment fraud in 1st half of 2024 alone. Victims are not careless — they are students, fresh graduates, and people urgently looking for work who encounter postings that look completely legitimate on the surface.

What frustrated me was that existing solutions either block entire platforms or rely on simple keyword matching. Neither approach understands why a posting is suspicious. I wanted to build something that could explain its reasoning the way a cautious human recruiter would: "this posting has no salary, no company profile, and four external links — that combination is almost always fraud."

That question became JobGuard.

What it does

JobGuard analyses any job posting for fraud signals and returns:

  • A fraud probability score from 0 to 100%
  • A verdict label: Legitimate, Caution, or Suspicious
  • A SHAP explanation showing the top 5 signals that drove the prediction, each with a plain-English description
  • A branch breakdown showing what the language model and the structural model each independently concluded

The system is intentionally strict. At the default threshold, every posting flagged as Suspicious has been genuinely fraudulent in testing. Postings that fall below the threshold but show warning signs are surfaced as Caution rather than silently passed.

How I built it

The core insight was that fraud lives in two places simultaneously: the language of the posting and its structure. A single model cannot capture both well. So JobGuard runs two independent branches and fuses their outputs.

Branch 1: Text (DistilBERT)

I fine-tuned distilbert-base-uncased on the concatenation of job title, description, and requirements. Standard cross-entropy loss fails on a dataset where only 4.5% of samples are fraudulent, so Iused Focal Loss:

$$\mathcal{L}_{focal} = -\alpha_t (1 - p_t)^\gamma \log(p_t)$$

where $\gamma = 2.0$ downweights easy negatives and forces the model to focus on hard-to-classify fraud cases. The model was exported to ONNX for fast CPU inference at serving time.

Branch 2: Structural signals (XGBoost)

I engineered 20 features capturing structural fraud patterns:

  • has_salary, has_company_logo, has_questions (direct fields)
  • location_vagueness (regex matching "anywhere", "remote only", "worldwide")
  • url_count (phishing link indicator)
  • all_caps_ratio (spam typography signal)
  • benefit_to_requirement_ratio (fake postings over-promise and under-specify)
  • company_profile_missing, description_length, and more

Class imbalance was handled via scale_pos_weight = n_negative / n_positive and early stopping on PR-AUC rather than accuracy.

Fusion

Both branch probabilities are combined via weighted late fusion:

$$score_{final} = w_{text} \cdot p_{bert} + w_{struct} \cdot p_{xgb}$$

Weights were tuned on the validation set ($w_{text} = 0.55$, $w_{struct} = 0.45$) rather than hardcoded.

Explainability

SHAP TreeExplainer runs on the XGBoost branch after every prediction. It returns per-feature contributions in probability space, which are ranked by absolute impact and surfaced to the user with direction (pushes toward fraud or legitimate) and a pre-written plain-English explanation per feature.

Stack

Backend: FastAPI, HuggingFace Transformers, XGBoost, SHAP, ONNX Runtime, Python 3.11

Frontend: React 18, TypeScript, Vite, Framer Motion, Recharts

Deployment: Docker, Hugging Face Spaces, GitHub Actions for CI/CD sync

Training: EMSCAD dataset (17,880 postings)

Challenges I ran into

Class imbalance was brutal. With only 4.5% positive rate, a naive model achieves 95.5% accuracy by predicting everything as legitimate. Accuracy is completely meaningless here. Switching to PR-AUC as the primary training metric and using Focal Loss for BERT and scale_pos_weight for XGBoost were both necessary to get the model to learn anything useful about fraud.

The Precision-Recall Tradeoff . My initial threshold was set at 0.85, which produced a precision of 1.0 — every single posting flagged as fraudulent was genuinely fraudulent, zero false alarms. However, recall sat at 0.51, meaning the model was silently missing nearly half of all fraud cases.It was a threshold problem. At 0.85 the model only speaks when it is extremely confident. That is useful in a context where a false alarm is costly — wrongly flagging a legitimate employer is a real harm. But it meant too many fraudulent postings were slipping through with a Caution label instead of Suspicious. I evaluated the full threshold sweep on the held-out test set. Lowering to 0.70 improved F1 by 9 points while keeping precision above 0.83 — a meaningful tradeoff. The threshold is stored as a separate artefact (xgb_threshold.joblib) and can be updated independently of the model weights, which meant no retraining was required. A single file update and server restart applied the change instantly.

SHAP and XGBoost probability outputs conflict. The model_output="probability" argument to TreeExplainer is not supported with the default feature_perturbation="tree_path_dependent". SHAP values are returned in log-odds space by default, which required careful handling to display meaningful impact numbers to users without misrepresenting what the values mean.

Deployment size. The full model artefacts exceed 1.2GB. Committing them directly to the Space repository was not practical. The solution was a separate Hugging Face model repository with snapshot_download at Docker build time, keeping the Space repo clean and the code deployable from GitHub via Actions.

Accomplishments that I'm proud of

  • Precision of 1.0 at the production threshold — every posting the model calls fraudulent is genuinely fraudulent
  • A fully explainable system where every prediction comes with ranked feature contributions in plain English, not just a number
  • An end-to-end pipeline from raw CSV to deployed Docker container, including automated sync from GitHub to Hugging Face Spaces on every push
  • A dual-branch architecture that separates language understanding from structural reasoning, making each branch independently interpretable and auditable

What I learned

Evaluation metric choice matters more than model choice. Swapping from accuracy to PR-AUC as the primary metric changed every training decision that followed. A model that looks good on accuracy can be completely useless on an imbalanced fraud detection task.

Explainability is not a feature you add at the end. Building SHAP into the architecture from the start forced cleaner separation between the two branches and made the structural feature engineering more deliberate. Every feature had to be justifiable to a non-technical user.

Late fusion is underrated. Training a learned fusion head is appealing but adds complexity and instability. Tuning scalar weights on the validation set produced competitive results with far less overhead.

What's next for Job Guard

  • Dynamic fusion weights based on input completeness — reduce BERT weight when description is missing, since sparse text input produces unreliable embeddings
  • Browser extension — analyse postings on LinkedIn, Indeed, and Rozee.pk without leaving the page
  • URL scraping — paste a job board link directly instead of copying text
  • Multilingual support — extend to Roman Urdu and Arabic using XLM-RoBERTa, covering job markets where English-only models completely fail
  • Feedback loop — let users flag incorrect predictions and build a correction dataset for periodic retraining
  • Lower the threshold dynamically — show a calibrated confidence interval rather than a single threshold, giving users more nuanced information about borderline cases

Built With

Share this project:

Updates

Submission history