BiasGPT

• 

  BiasGPT-homepage

• 

  BiasGPT-Query

BiasGPT: An Automated Demographic Bias Detection System

Inspiration

The idea for BiasGPT was born out of a desire to explore and mitigate demographic bias in AI-generated content. As language models become more integrated into products and systems, it’s essential to understand how subtle shifts in race, gender, or other identity traits can influence the tone or content of generated responses. I wanted to build a tool that would not only detect these shifts but also learn from them over time.

What I Built

BiasGPT is an end-to-end system that:

• Takes a prompt and generates a demographically-swapped version using spaCy and a custom dictionary.
• Scores the "tone shift" between the original and swapped prompt using OpenAI’s GPT-3.5.
• Labels responses based on the shift and stores them in a dataset.
• Retrains a DistilBERT classifier nightly to improve its bias detection capabilities.
• Includes a React frontend and a FastAPI backend for live interaction.

Tech Stack:

• Frontend: React + Tailwind CSS (Next.js)
• Backend: FastAPI
• ML/NLP: Hugging Face Transformers, spaCy, OpenAI API
• Automation: Cron (macOS LaunchAgent fallback), bash scripts
• Model training: PyTorch, Scikit-learn
• Data handling: pandas, CSV pipelines

What I Learned

• How to automate nightly ML pipelines with cron and bash scripting
• How to tokenize, train, and save Hugging Face transformer models
• Managing environment variables securely for API access
• Creating modular, debuggable code for complex pipelines
• Using launchctl as a fallback to cron on macOS for scheduled jobs

Challenges I Faced

• Setting up cron jobs on macOS was more complex than expected due to SIP (System Integrity Protection)
• Loading large spaCy models and dealing with slow runtime at scale
• Ensuring .env variables loaded correctly in every context (cron, shell, etc.)
• Resolving issues with model serialization (e.g., mismatched keys in state_dict)
• Detecting subtle differences in tone and formalizing that into a reliable metric
• Avoiding submodule conflicts in GitHub and ensuring a clean repo structure

Current Status & Room to Grow
BiasGPT is intentionally in its infancy: the classifier is trained on a very small, synthetic dataset, so its predictions are still rudimentary. What matters most at this stage is the automated pipeline—every night the system generates new swapped‑prompt pairs, scores them, merges them into the master dataset, and retrains the model. As that dataset grows from a few dozen examples to thousands, the classifier’s F1 score will steadily improve and the system will evolve from a proof‑of‑concept into a genuinely smart bias‑detection engine.

What’s Next

• Improve demographic swaps to be more context-aware
• Use more advanced classifiers for tone and sentiment
• Support batch uploads and human feedback loops in the UI
• Extend support to other biases (e.g., age, socioeconomic status)

Built With

• cron
• dotenv
• fastapi
• github
• huggingfacetransformers
• numpy
• openaiapi
• pandas
• python
• pytorch
• react
• spacy
• tailwindcss