Bias

Bridge the gap for eqaulity one detection at a time.

Comment

Inspiration

Racial inequality is a real and prevalent problem that affects almost all minorities. Don’t just take my word for it according to www.kff.org Three in ten Black (31%) and ASIAN adults (28%) and about one in four Hispanic (26%) and Asian adults (25%) say they experienced at least two of these types of discrimination at least a few times in the past year, all higher than the share of White adults who say so (18%). Not only that but minorities are under-reported in the news and bias of one races supremacy over another is still prevalent. In fact, according to the national institute of health A majority of black adults reported experiencing discrimination in employment (57 percent in obtaining equal pay/promotions; 56 percent in applying for jobs)

What it does

My final solution is a customtkinter frontend that is able to detect bias given by input in a textbox. The user then gets a bias score were higher is lower. This feature allows the user to see the models confidence of its prediction. The program also then concludes weather the text is biased or not. Next, the program gives unbiased articles as recommendations to check out. Lastly, the program gives recommendations for fixing bias in the text in bullet points.

How we built it

I used pytorch for the NLP (to specify the exact model I used was an STLM). The NLP gets an input form the user. Next it determines whether the text has bias or doesn't have any bias. After that if it has bias another ai this time a LLM (Llama) goes over the text and gives recommendations to remove bias. Another feature then takes a summary of the text and converts it to a likely headline. Which is added to an fstring which is the website link that will be given to the user. Lastly NLP also gives a confidence meter which is displayed to show the user how biased the text is.

Challenges we ran into

Trying different tranformers such as BERT and variations of BERT. However they wouldn't work. Debugging issues and increasing accuracy. I had to combat data overfitting, class imbalance and utilizing weight optimizers.

Accomplishments that we're proud of

Creating an NLP model form scratch and a complicated one that improved my knowledge of machine learning and advanced math topics.

What we learned

How to create NLP's, different techniques to increase accuracy and some of the math behind machine learning models.

What's next for Bias

A better NLP using transformers to increase accuracy and a bigger dataset than just hugging face. Also experimenting with different weight optimizers would be great to find the optimal weight optimizers for accuracy.

Built With

  • adamw
  • bcewithlogits
  • berttokenizer
  • customtkinter
  • llama
  • pytorch
  • re
  • seaborn
  • sklearn
  • tqdm
  • webrowser
  • xaiver-uniform
Share this project:

Updates