FairLoanAudit

• 

  gan_vs_xgb_disagreement

• 

  approval_distribution

• 

  bias_visualization

• 

  flip_barplot

• 

  shap_barplot_xgb

• 

  shap_summary_xgb

Inspiration

As AI systems increasingly influence life-changing decisions like loan approvals, we became concerned about how invisible bias can persist or even worsen in model outputs. We asked: How can we ensure not just accurate, but fair predictions for everyone — especially underrepresented groups? This led us to experiment with GANs not for generation, but for fairness benchmarking.

What it does

FairLoanAudit is a tri-model system that predicts loan approvals and audits them for demographic bias. It includes:

• Logistic Regression for interpretability
• XGBoost for performance and explainability (SHAP)
• A GAN trained on White Male approvals to simulate an unbiased fairness benchmark

The system performs fairness audits like UFD (Unjustified False Denial) detection, counterfactual flip testing, and produces both fairness-informed and production-ready predictions.

How we built it

• Cleaned and encoded demographic + financial loan data
• Trained Logistic and XGBoost classifiers using scikit-learn and XGBoost
• Trained a PyTorch-based GAN Generator using approved White Male records as banchmark
• Audited denial patterns with UFD heatmaps and flip tests
• Used SHAP to interpret feature importance in XGBoost
• Generated predictions from all models on test data
• Compared GAN fairness scores to XGBoost outcomes for bias identification

Challenges we ran into

• Designing a GAN that could simulate ideal approvals without collapsing
• Ensuring fair comparison between GAN scores and model outputs
• Mapping demographic flip tests reliably while keeping financial features untouched
• Structuring UFD and flip tests in a reproducible and visualizable way
• Balancing fairness auditing and prediction performance in a clear narrative

Accomplishments that we're proud of

• Used GANs in a novel way: as fairness auditors and counterfactual predictors
• Achieved a full audit pipeline: UFD, flip tests, SHAP, demographic breakdowns
• Revealed that XGBoost denied 581 cases that the GAN considered eligible
• Visualized demographic bias with clear, interpretable heatmaps and barplots
• Produced a fairness-informed prediction file (gan_prediction.csv) and bias report

What we learned

• Bias lives not just in data, but in historical patterns that models replicate
• GANs can be fairness tools, not just image generators
• Explainability (like SHAP) is critical when making models trustworthy
• Counterfactual thinking makes fairness measurable and visible
• Transparency requires multi-perspective auditing, not just one metric

What's next for FairLoanAudit

• Implement fairness-aware reweighting and adversarial debiasing techniques
• Integrate LIME explainability alongside SHAP for model comparisons
• Build a dashboard for live fairness analysis and model auditing
• Release the GAN benchmarking module as an open-source fairness toolkit
• Apply the pipeline to other domains (e.g. hiring, healthcare, insurance)

Built With

• python