-
-
Starting Page
-
Explainability
-
Overview
-
Audit Report
-
Intersectionality
Inspiration
Artificial intelligence is increasingly used to make high-impact decisions in hiring, lending, healthcare, education, and public services. While these systems improve efficiency, they can also inherit and amplify historical bias hidden within training data.
Well-known examples, such as biased hiring algorithms and unfair criminal risk assessment systems, highlight the importance of auditing AI before deployment. Existing fairness toolkits are powerful but primarily target machine learning practitioners, often exposing statistical outputs that are difficult for non-technical users to interpret.
EquiLens-AI was inspired by a simple question:
What if anyone—an NGO worker, teacher, policymaker, or student—could audit AI systems for fairness without needing a data science degree?
Our goal was to democratize AI fairness by combining explainable AI, fairness metrics, and natural-language explanations into one accessible platform.
What it does
EquiLens-AI is an open-source AI fairness auditing platform that detects, explains, and helps mitigate algorithmic bias in datasets.
Users simply upload a CSV dataset, select a target column and one or two sensitive attributes, and EquiLens-AI generates a complete fairness report.
Features
- Upload CSV datasets
- Compute fairness metrics:
- Statistical Parity Difference (SPD)
- Disparate Impact (DI)
- Equalized Odds (EO)
- SHAP-based explainability
- Intersectional fairness analysis
- Audience-aware AI explanations (NGO, Student, Policymaker)
- Interactive "What-if" simulator
- Downloadable PDF audit reports
- Compliance guidance (EEOC, GDPR, EU AI Act)
Fairness Metrics
Statistical Parity Difference
$$ SPD = P(\hat{Y}=1|A=a)-P(\hat{Y}=1|A=b) $$
where
- $\hat{Y}$ is the predicted outcome
- $A$ is the sensitive attribute
Disparate Impact
$$ DI=\frac{P(\hat{Y}=1|A=a)}{P(\hat{Y}=1|A=b)} $$
Values below
$$ DI < 0.8 $$
may indicate potential discrimination under the EEOC 4/5ths rule.
Equalized Odds
We compare both
$$ TPR $$
and
$$ FPR $$
between protected groups.
A large difference indicates unequal model performance across demographics.
How we built it
Backend
- FastAPI
- Python 3.12
- Pandas
- NumPy
- Scikit-learn
- SHAP
- ReportLab
Frontend
- HTML
- CSS
- JavaScript
- Chart.js
AI Layer
- Gemini 2.5 Flash
Workflow
- Upload dataset
- Parse CSV
- Train Random Forest model
- Compute fairness metrics
- Generate SHAP explanations
- Perform intersectionality analysis
- Generate AI explanations
- Produce PDF audit report
The backend exposes REST APIs that power an interactive dashboard, while Gemini converts statistical outputs into audience-specific explanations that anyone can understand.
Challenges we ran into
One of the biggest challenges was implementing fairness metrics that worked reliably across different datasets while supporting both numerical and categorical labels.
Building intersectionality analysis was especially challenging because many subgroup combinations contain very few samples. We introduced minimum sample thresholds to prevent misleading conclusions.
Another challenge was balancing explainability and performance. SHAP can be computationally expensive, so we optimized our workflow to keep response times practical while still providing meaningful explanations.
Finally, translating statistical fairness metrics into plain language without oversimplifying the results required careful prompt engineering and fallback mechanisms when the AI API was unavailable.
Accomplishments that we're proud of
We're proud that EquiLens-AI delivers much more than a fairness calculator.
Highlights include:
- End-to-end AI fairness auditing
- Real intersectionality analysis
- SHAP explainability
- Audience-specific AI explanations
- Interactive What-if simulator
- Professional PDF audit reports
- Automatic dataset column detection
- Open-source implementation
- Clean, intuitive dashboard for non-technical users
Most importantly, we built a platform that lowers the barrier to responsible AI by making fairness auditing accessible beyond data scientists.
What we learned
This project taught us that responsible AI is as much about communication as it is about algorithms.
While implementing fairness metrics required a solid understanding of machine learning, the real challenge was presenting those results in a way that users could actually understand and act upon.
We also gained valuable experience with:
- FastAPI backend architecture
- Explainable AI using SHAP
- Fairness evaluation methodologies
- REST API design
- Interactive data visualization
- Prompt engineering for AI-assisted explanations
- Building production-ready ML applications
What's next for EquiLens-AI
Our roadmap includes several major improvements:
- Additional fairness metrics
- Bias mitigation algorithms
- Real-time monitoring for deployed models
- Image, text, and multimodal model support
- Team collaboration workspaces
- Automated compliance reporting
- Cloud deployment with authentication
- Mobile-responsive interface
- Model versioning and audit history
- API integrations with popular ML workflows
Our long-term vision is to make EquiLens-AI a comprehensive fairness platform that enables organizations to build trustworthy AI systems from development through deployment.
Log in or sign up for Devpost to join the conversation.