Inspiration
UN SDG 5 highlights various challenges faced by women and girls all around the world. My solution aims to tackle the problem of women and girl’s safety highlighted in goal 5.2 which talks about eliminating all forms of violence against women and girls.
Social media has evolved beyond being merely a personal tool; it now serves as marketplace for businesses, platform for community-building, and a valuable learning resource. Women and girls, particularly those who are popular or influential, often face unwarranted hate and harassment through direct messages or public comments. These messages can come in various forms, including text, images, audio, and video, which impacts their mental and psychological health, and leaves them feeling helpless and vulnerable.
The most effective way to safeguard from online hate is to minimize exposure. By ensuring that hateful messages are not visible in the first place, women can protect their mental and emotional well-being. This approach involves proactively choosing not to engage with negativity, thereby maintaining a barrier against harmful content.
What it does
Solution will analyze and categorize messages received in any format (text, audio, video and image), using multimodal model gemini-1.5-flash into 6 main categories-
- Verbal Abuse, Bullying and Mental Harassment
- Stalking and Doxing
- Sexual Harassment
- Hate Speech and Discrimination
- Online grooming
- Instigating violence and self-harm
If the message received was malicious, it will blur the content to the receiver. Receiver will have the option to View the message, Report the message and Get Help. If the message was categorized as non-malicious, it will be presented as is to the receiver.
How we built it
Tools that I used to build this solution are -VS Code, Google AI Studio for POC, Multimodal model gemini-1.5-flash API for classification. I used python as programming language.
Challenges we ran into
Since I am a data scientist by profession and not a software developer, I faced challenges with the front end code.
Accomplishments that we're proud of
I am proud that I was able to build a solution which has potential to help women and girls all around the world. Shielding from online hate can play a significant role in promoting gender equality by fostering an environment where women can participate fully and confidently in online spaces. It can help them to leverage the full potential of social media, allowing them to build networks, careers, and advocate for issues that matter to them without the fear of harassment.
What we learned
I learned about the UN SDG 5 and how women and girls face so many challenges all around the world. I also learned about the Gemini API implementation and AI Studio features.
What's next for Online Harassment Shield
- Adding this feature in actual social media platforms would help create a safe space for everyone.
- The feature to "Report" and "Get Help" can be enhanced to redirect to actual report and help/counselling portals.
- Hide message feature can be enhanced to disable "View Message" option for underage users.
- More categories of harassment types can be included, currently only 6 types are defined in the code.
- To predict with higher accuracy and consistency, model can be fine-tuned in future.
Log in or sign up for Devpost to join the conversation.