Inspiration

We were inspired by a common problem faced by students: cyberbullying and harmful communication can affect a student's confidence, emotional well-being, and willingness to seek help. In many cases, the issue is noticed only after it becomes serious. We wanted to build a platform that can identify harmful content early and connect it with appropriate human support, especially counselors and teachers.

What it does

AI-Based Cyberbullying and Harmful Content Detection Platform analyzes student messages and supported submissions to identify potentially harmful content such as harassment, insults, threats, and exclusion. It uses AI to assess the content and its severity. When harmful content is detected, the system can create a safety case or alert for authorized counselors or staff to review. Role-based dashboards allow students, teachers, counselors, administrators, and principals to access the features relevant to them.

How we built it

We built the platform using React 18 and Vite for the frontend and Java Spring Boot for the backend. The system has two backend services: an Auth & User Service and a Main Core & Safety Service. We use JWT and Spring Security for authentication and role-based access. MySQL/TiDB Cloud handles user-related data, while MongoDB/MongoDB Atlas stores application and safety-related data. For harmful-content analysis, we use Google Gemini 1.5 Flash, supported by a local Java rule-based NLP analyzer as a fallback.

Challenges we ran into

One of our main challenges was detecting harmful content when the meaning of a message depends on its wording and context. We also had to handle different types of harmful content and decide how to represent its severity. Integrating AI analysis with the backend while maintaining secure authentication and role-based access was another challenge. We also had to make sure that the system could continue basic analysis when the external AI service was unavailable.

Accomplishments that we're proud of

We are proud of building a working end-to-end platform focused on student safety and mental well-being, rather than only creating a standalone text classifier. We integrated AI-based content analysis into actual student communication and submission workflows and connected detected issues to a safety-case and counseling workflow. We also implemented different experiences for students, teachers, counselors, administrators, and principals.

What we learned

We learned that solving a real-world AI problem involves much more than training a model. We learned how to integrate AI with a full-stack application, work with multiple databases, implement authentication and role-based access, handle API failures, and design a workflow where AI supports human decision-making rather than replacing counselors. We also learned the importance of testing, validation, and responsible handling of sensitive student information.

What's next

Our next step is to make the platform more capable of supporting early intervention and student well-being. We plan to explore multimodal detection, including harmful content in images and memes, and improve detection of Tamil, Tanglish, and other code-mixed student communication. We also want to improve explainability so counselors can understand why content was flagged, while continuing to prioritize privacy, responsible AI use, and human review.

Built With

  • axios-backend:-java
  • docker
  • gemini1.5flash
  • github
  • java
  • jwt
  • jwt-databases:-mysql/tidb-cloud-and-mongodb/mongodb-atlas-ai:-google-gemini-1.5-flash-ai-fallback:-java-rule-based-nlp-analyzer-deployment:-docker
  • mongodb
  • mysql
  • react
  • render
  • spring-boot-authentication:-spring-security
  • springboot
  • tidbcloud
  • vercel
  • vite
Share this project:

Updates

Submission history