TrustLens AI

This is the TrustLensAI which is a moderation assistant app which helps moderators to detect harmful, restricted comments faster.

Comment

Inspiration

My app TrustLensAI was inspired by the challenges faced by moderators in managing the harmful online content across communities. While it seems basic, these comments can degrade the engagement and affect mental health of an individual. Moderators often deal with toxic comments, hate speech and harassment. As student interested in AI and safety of online community, I wanted to create a tool which can assist moderators using artificial intelligence to identify harmful behavior, analyze comments and provide several moderation insights in a faster way.

What it does

The app monitors the Reddit comment activity by Devvit triggers and analyzes the comment for harmful behavior in the platform like toxicity, threats and unsafe language platforms. The system does provide moderators with AI-generated moderation recommendations, sentiment analysis and moderation insights through interactive dashboard and records the reports in the report interface.

How we built it

We built the TrustLensAI as a combination of a flask-based AI moderation dashboard and a Reddit devvit integration. For frontend part of app I used HTML, CSS and Javascript to create an interactive moderation dashboard and reporting interface. I designed the home page to describe the purpose of the system and then I extended the home page with the dashboard page and the report page to analyze the comments, view the scoring and then see the analysis data using the report page. For backend part, I used Python and Flask which manages the routing, processes moderation requests and connects the frontend part with the AI analysis engine. The AI analysis system evaluates the comments, generating the moderation insights such as confidence level, toxicity score and sentiment classification and gives moderator suggestions. I integrated the developer platform of the Reddit and Devvit to connect the project with the Reddit environment. The Devvit component was then used to structure the Reddit app workflow and simulate the how moderators use AI-assisted moderation tools with Reddit communities. The reports and analytics section was designed to dynamically display the moderation insights, flagged activity, harmful keyword detection and moderation timelines which helps moderators t better understand the behavior of community patterns.

Challenges we ran into

One of the biggest challenges faced during the development was the integration of the multiple technologies into a single workflow. Since TrustLensAI combined the flask backend with AI, fronted and Reddit Devvit integration, the communication and deployment was a huge issue. My initial attempt was using the Github and Render but the Render deployment gave several issues because it couldn't deploy the apps which had the heavy machine learning modules. Secondly, the external devvit app domain approval process is really lengthly and it is still pending , however, the moderation integration pipeline has already been fully implemented and successfully tested within our development environment.

Despite these challenges, the process helped us to improve the understanding of the full-stack development, the AI integration in our workflow, debugging the workflow and moderation-focused app design.

Accomplishments that we're proud of

I am proud of building an AI-powered moderation platform which combines the frontend development, integrates the backend, AI-assisted analysis and then integrates the Reddit Devvit into a single workflow. I was able to create a responsive moderation dashboard which is capable of analyzing the comments, along with gathering the moderation insights and dynamically updating the moderation reports. Most importantly , I feel proud that I could transform the project from an initial concept to a fully functional app which demonstrates how AI can be used to support safer online communities.

What we learned

Through this project, I learned full-stack development, AI system integration and debugging workflows. We improved our understanding of the flask backend development, frontend synchronization using javascript and dynamic report generation. I gained hands-on experience working on the Reddit Developer Platform. Hence, this project helped us to better understand the practical challenges of integrating the AI systems into real-world moderation environments, including handling the consistency of data, the usability of interface and deployment limitations.

What's next for TrustLens AI

In future, I plan to expand the TrustLens AI into a more advanced real-time moderation assistant which monitors the live Reddit comment and automatically detects the harmful behaviors. I want to improve the AI analysis engine by training the model from scratch so moderators can directly interact with the AI-powered moderation tools within the reddit communities. Another future goal is to strengthen the Devvit integration pipeline so that moderators can directly interact with the AI-powered tools within Reddit communities. For long term, my vision is to transform the TrustLensAI into a scalable platform which helps the online communities to maintain safer ,healthier and more organized environments.

Built With

Share this project:

Updates