TRACE: Tracking Risk Across Comments and Evaluation

• 

  More contents in the dashboard

• 

  Team logo

• 

  Dashboard 2nd screen

• 

  Dashboard 1st screen

• 

  AI architecture

Inspiration

Our team, Trinetra.ai, is named after the mythological third eye that sees what ordinary sight cannot. The idea for TRACE came from a simple observation: while social media platforms can remove harmful content, they often fail to recognize harmful behavior that develops gradually over time. Many online predators do not rely on obviously abusive messages. Instead, they build trust through repeated interactions across days, weeks, or even months. If one child blocks them, they can simply move on to another. Existing moderation systems often treat each interaction independently, meaning no behavioral memory follows the sender. We wanted to build a system that could identify these patterns before serious harm occurs.

What it does

TRACE is an AI-powered early warning system that helps Trust and Safety teams identify potentially harmful behavior toward children before it escalates. Rather than evaluating users based on individual messages, TRACE builds a behavioral profile from repeated interactions across time and across multiple child accounts. This allows analysts to identify patterns such as repeated targeting, trust-building behavior, and escalation that may not be visible through traditional moderation approaches. The system presents evidence-backed insights and recommendations through an analyst dashboard while keeping all enforcement decisions under human control.

How we built it

We built TRACE as a multi-stage AI pipeline supported by custom synthetic datasets and an analyst-facing dashboard. To overcome the lack of publicly available child-safety datasets, we developed a Python-based synthetic data generation framework capable of producing realistic interaction records and sender-level behavioral histories. We then trained and integrated multiple AI models to analyze risk signals and behavioral patterns before presenting the results through a Streamlit dashboard designed for Trust and Safety workflows. The final system combines data generation, model inference, behavioral analysis, and explainable reporting into a single end-to-end pipeline.

Challenges we ran into

One of the biggest challenges was that grooming behavior is intentionally designed to appear harmless. Individual messages often look friendly and supportive, making traditional content moderation ineffective. This forced us to shift our focus from detecting harmful words to identifying behavioral patterns across time and across victims. Another challenge was the lack of publicly available datasets. To address this, we designed a synthetic data generation pipeline that could realistically model interaction patterns while remaining entirely fictional and privacy-preserving. Finally, integrating multiple AI components and a dashboard within a limited hackathon timeline required careful coordination across parallel development streams.

Accomplishments that we're proud of

We are proud of creating a working prototype that shifts moderation from content-based detection to behavior-based risk assessment. Within a hackathon timeframe, we designed synthetic datasets, trained models, built an analyst dashboard, and delivered a complete end-to-end workflow. We also ensured that every recommendation remains explainable and reviewable, allowing human analysts to understand the evidence behind each assessment rather than relying on black-box outputs.

What we learned

This project taught us that harmful online behavior is often a behavioral problem rather than a language problem. The most concerning actors rarely reveal themselves through a single message; they reveal themselves through patterns that emerge over time. We also learned that explainability is essential for Trust and Safety applications. Analysts need to understand why a recommendation was generated before they can confidently act on it. Building transparency into the system from the beginning was just as important as building the AI itself.

What's next for TRACE: Tracking Risk Across Comments and Evaluation

Our current prototype operates on synthetic data. The next step is validating the approach with real-world partners and experts while ensuring strong ethical, legal, and privacy safeguards. We also want to explore privacy-preserving approaches such as federated learning, allowing platforms to improve detection capabilities without sharing sensitive user data. Long term, we envision TRACE as an infrastructure layer that can be integrated into any platform hosting child-facing content, helping Trust and Safety teams identify repeat harmful behavior before it escalates.

Built With

• bert-(bert-base-uncased-fine-tuned)
• gemma-llm
• jsonl
• python
• streamlit
• synthetic-data(generated-using-python-program)