TellMom

TellMom training process
TellMom prototype

Inspiration

Online games have become an important part of children's social lives, allowing them to communicate, make friends, and participate in virtual communities. However, these platforms also expose children to risks such as online grooming, cyberbullying, scams, and the accidental sharing of personal information.

Recent reports have highlighted the scale of this challenge. In 2023, Roblox reported more than 13,000 child exploitation incidents to the National Center for Missing and Exploited Children, showing that online safety remains a significant concern for platforms with large numbers of young users. Harmful interactions often begin with seemingly harmless conversations, making them difficult for parents and traditional moderation systems to detect early.

Therefore, we created TellMom to act as an intelligent early-warning system for families. By using AI-powered conversation analysis, TellMom helps identify potential threats before they escalate, giving parents the information they need to protect their children while respecting their privacy and independence using COPPA regulations.

What it does

TellMom is an AI-powered child safety platform that monitors in-game communications and activities for potential risks. When a supported game is launched, the application collects available chat logs and analyzes them for suspicious behavior such as grooming attempts, requests for personal information, cyberbullying, or accidental exposure of family details.

The system performs a quick keyword-based scan for immediate threats and an AI-powered contextual analysis for more sophisticated detection. If suspicious activity is identified, parents receive an alert with supporting evidence, while the child can be warned in real time. Future versions will also use computer vision to analyze screenshots and detect unsafe visual activities that text logs alone cannot capture.

How we built it

We developed TellMom as a distributed AI-powered child safety platform consisting of three main components: an Android monitoring application, a backend analysis server, and an AI-based threat detection engine.

The Android application acts as the data collection layer. When a supported game such as Roblox is launched, the application initiates a monitoring process that gathers available communication logs generated during gameplay. The raw logs are filtered to extract relevant chat messages while removing unrelated system events and metadata. This preprocessing stage reduces noise and prepares the data for analysis.

The processed conversations are transmitted through secure HTTPS API calls to a backend server. To maintain data privacy and support COPPA compliance, only the minimum information required for threat analysis is transmitted and stored. All communications between the mobile application and backend services are encrypted during transmission.

The backend performs a two-stage detection pipeline.

The first stage is a rule-based threat scanner designed for low-latency detection. This scanner uses keyword matching, pattern recognition, and predefined safety rules to identify common indicators such as requests for personal information, attempts to move conversations to external platforms, age-related questioning, or potentially abusive language. This stage provides near real-time alerts with minimal computational overhead.

The second stage is an AI-powered contextual analysis system. Conversation segments are processed by a natural language processing (NLP) model trained to recognize patterns associated with online grooming, cyberbullying, manipulation tactics, and accidental exposure of sensitive family information. Unlike simple keyword filtering, the AI model evaluates conversational context, message history, and semantic meaning to detect suspicious interactions that may develop gradually over time.

Detection results are assigned risk scores based on confidence levels generated by the analysis pipeline. When the risk score exceeds predefined thresholds, the backend generates evidence-based alerts that are sent to parents through the mobile application. The alert includes the relevant conversation snippets and the specific risk category identified by the system.

The platform architecture was designed around cybersecurity principles, particularly the CIA Triad (Confidentiality, Integrity, and Availability). Data confidentiality is maintained through encryption and restricted access controls. Data integrity is protected through secure API communication and validation mechanisms. Availability is supported through a scalable backend architecture capable of handling multiple analysis requests simultaneously.

For future development, we designed a hybrid AI architecture that combines text analysis with computer vision. Periodic gameplay screenshots will be processed using Optical Character Recognition (OCR) and image-captioning models to extract both visible text and gameplay context. The extracted information will then be correlated with chat conversations, allowing the system to identify threats that may not be detectable through text logs alone.

Challenges we ran into

One of the most significant technical challenges was obtaining reliable and relevant conversation data from multiplayer gaming environments. Roblox chat logs often contain messages from all players within a server session, regardless of whether they are directly interacting with the monitored child. This creates a substantial amount of irrelevant data that can reduce detection accuracy and increase false-positive rates. To address this issue, we designed filtering mechanisms that attempt to isolate meaningful conversational exchanges before analysis.

Another major challenge involved detecting online grooming behavior. Grooming is rarely characterized by explicit language during its early stages. Instead, predators frequently use trust-building techniques, gradual relationship development, emotional manipulation, and indirect questioning. Traditional keyword-based filtering systems struggle to detect these behaviors because individual messages may appear harmless when viewed in isolation. This required us to explore contextual AI analysis capable of evaluating message sequences and conversational patterns over time.

Dataset quality also presented a challenge. Publicly available datasets containing examples of grooming conversations are limited due to privacy concerns and legal restrictions surrounding child safety data. This creates difficulties in training machine learning models that can generalize effectively to real-world interactions. Significant effort was required to identify suitable datasets, preprocess training data, and design a strategy for continuously updating detection knowledge as online threats evolve.

Privacy and regulatory compliance introduced additional engineering constraints. Since the platform handles information involving minors, every component of the system had to be designed with data minimization principles in mind. We needed to balance effective threat detection with strict limitations on data collection, storage, and processing in order to align with COPPA requirements and modern privacy standards.

From a systems perspective, balancing detection accuracy and performance was another challenge. Real-time safety monitoring requires low-latency analysis, but advanced AI models can introduce computational overhead and increase response times. To solve this, we implemented a hybrid architecture that combines lightweight rule-based scanning for immediate threat identification with deeper AI analysis for higher-confidence contextual detection.

Looking toward future development, computer vision integration presents its own technical challenges. Capturing screenshots too frequently can negatively impact device performance and battery consumption, while capturing too infrequently may cause important events to be missed. We therefore began exploring event-driven image capture strategies that increase monitoring frequency only when suspicious activities are detected through text analysis, improving both efficiency and detection coverage.

Accomplishments that we're proud of

We successfully designed an end-to-end MVP that can collect game communication data, analyze conversations for potential threats, and notify parents when suspicious activity is detected.

We're especially proud of creating a privacy-first architecture that prioritizes child protection without requiring invasive monitoring. Our solution focuses on evidence-based alerts rather than constant surveillance.

We also established a clear roadmap toward a hybrid AI system that combines text analysis and computer vision, allowing the platform to detect risks that traditional parental control applications may miss.

What we learned

Throughout this project, we learned that online child safety is not just a technical problem but also a privacy and trust challenge. Building effective protection requires balancing security, compliance, user experience, and ethical AI practices.

We also gained hands-on experience with AI-based text classification, dataset preparation, mobile-to-server architecture, and the complexities of detecting harmful online behavior in real-world conversations.

Most importantly, we learned that early intervention can make a significant difference in preventing online threats before they escalate into real-world harm.

What's next for TellMom

Our next objective is to transform TellMom from a text-based monitoring solution into a comprehensive AI-powered child online safety platform capable of understanding both communication and behavior across multiple digital environments.

The first major enhancement is the integration of a computer vision pipeline. While text analysis can identify many threats, harmful activities often occur through visual content, game interactions, profile information, or user-generated assets that are not captured within chat logs. To address this limitation, TellMom will periodically capture gameplay screenshots and process them using Optical Character Recognition (OCR), image classification models, and multimodal AI systems. This will allow the platform to detect visual indicators of cyberbullying, inappropriate content, grooming attempts, and other potentially harmful activities that text-only analysis may miss.

We also plan to develop a hybrid detection engine that combines chat analysis and computer vision into a unified risk assessment framework. By correlating conversation data with gameplay behavior and visual context, the system will be able to generate more accurate threat assessments and significantly reduce false positives. For example, suspicious conversations identified through chat logs can trigger more frequent visual monitoring, enabling the system to gather additional evidence before generating alerts.

Another major focus is expanding the platform's detection capabilities beyond online predator identification. Future AI models will be trained to recognize cyberbullying, toxic behavior, social manipulation tactics, scams, phishing attempts, and accidental disclosure of personal or family information. This broader threat coverage will allow TellMom to protect children from a wider range of online risks commonly encountered in gaming and social platforms.

To improve detection accuracy over time, we aim to implement a continuously updated AI training pipeline. New threat patterns, emerging slang, and evolving online grooming techniques can be incorporated into future model updates through curated datasets and ongoing evaluation processes. This adaptive learning approach will help the platform remain effective as online threats continue to evolve.

Platform expansion is another key milestone. While the MVP focuses on Roblox, our long-term architecture is designed to support additional gaming and communication platforms. Future integrations may include multiplayer games, social networking applications, messaging platforms, and virtual communities frequently used by children and teenagers. A modular data collection framework will allow new platforms to be added without redesigning the core detection engine.

We also plan to strengthen privacy-preserving AI capabilities. Future versions will explore on-device processing and edge AI techniques, allowing sensitive data to be analyzed locally whenever possible. This approach reduces data transmission requirements, improves user privacy, and helps maintain compliance with child safety and data protection regulations.

Finally, our long-term vision is to create an intelligent digital guardian that proactively helps families navigate online environments safely. Rather than acting solely as a monitoring tool, TellMom will evolve into a real-time risk prevention system capable of identifying threats early, educating children about online safety, and providing parents with meaningful insights while respecting children's privacy and independence.