Thumbnail Image
Integrating SafeStream
No SafeStream Integration
After Integrating SafeStream

SafeStream: Responsible AI for Personalized Content Filtering on TikTok Reels

In today's digital age, content consumption is vast and varied, often exposing users to material they might find distressing or inappropriate. SafeStream harnesses the power of responsible AI to give users control over their viewing experience on TikTok Reels. By enabling users to filter out content based on their preferences, SafeStream ensures a safer and more enjoyable experience, promoting mental well-being and inclusivity.

Imagine being able to avoid political debates, offensive jokes, or graphic content simply by setting your preferences. Our state-of-the-art generative AI models analyze each TikTok Reel in real-time, identifying content categories like political views, offensive material, or sensitive topics. When a match is found, users receive a non-intrusive alert, allowing them to skip or snooze the content, preventing potential trauma or distress.

SafeStream empowers users with informed choices, fostering a healthier digital environment. With features tailored to diverse needs, from avoiding political extremism to skipping mature themes, our system is inclusive and adaptable. This project showcases TikTok as a pioneer in responsible AI, leading the way in user-centric content moderation.

By implementing SafeStream, TikTok can set a new standard for social media platforms, demonstrating a commitment to user well-being and the ethical use of AI. Join us in making the internet a safer, more comfortable place for everyone.

Key Features Highlighted

User Control: Allows users to set detailed preferences for the content they wish to avoid.
Real-Time Analysis: Uses state-of-the-art generative AI models to analyze TikTok Reels and match them with user preferences.
Informed Choices: Provides non-intrusive alerts and options to skip or snooze content.
Promotes Mental Well-Being: Protects users from distressing or unwanted content, enhancing their digital experience.
Inclusive and Adaptable: Caters to diverse user needs, ensuring a broad range of content preferences are covered.

About the Project

Inspiration

The inspiration behind SafeStream was the growing concern over the impact of distressing and inappropriate content on mental well-being. As avid users of social media platforms, we realized the need for a solution that empowers users to control their content consumption and avoid potentially harmful material.

What We Learned

Throughout the development of SafeStream, we learned the intricacies of integrating advanced AI technologies for real-time content analysis and filtering. We also gained insights into user preferences and the importance of providing a customizable and inclusive experience.

How We Built the Project

We created two services:

Content Analysis Service:
- Built using Flask and state-of-the-art generative AI models.
- Preprocesses video, audio, and text to extract multimodal information.
- Analyzes the content to understand its context and labels it according to the user's content preferences.
User Preference Service:
- Developed on the MERN stack with Firebase.
- Provides a user-friendly interface for setting content preferences.
- Displays warnings and updates the recommendation engine based on user choices.

Technical Architecture

Frontend: React.js, Redux, Firebase for authentication and real-time database.
Backend: Node.js, Express, MongoDB for user data storage.
AI Models: Integrated using Flask, leveraging state-of-the-art generative AI models for content analysis.
Processing Pipelines: Video, audio, and text preprocessing for extracting contextual information.
Alert System: Real-time pop-ups and notifications to inform users about content matching their exclusion preferences.
Generative Model APIs: OpenAI Whisper model for speech recognition, Anthropic Claude model for prompt engineering

Design Thinking and Methodology

User Interaction Flow

Frontend Service and User Preferences:
- Users interact with a frontend service where they access their user profile settings.
- Within these settings, users select content preferences read from a content_preferences file.

The Content Preferences JSON File was created to serve as a robust framework for users to manage their content consumption effectively. It enables users to toggle preferences across various categories, such as political content, sexuality and gender issues, offensive material, and more. Each category is further subdivided into specific subcategories, ensuring granular control over the type of content users encounter. Link to View: https://drive.google.com/file/d/1lc1_KH6YGK1tvzlrtO0r-nIp_XQIvPk9/view?usp=sharing

Video Upload Triggers Backend Service

Backend Service Workflow (Flask Application):
- Video Upload Handling:
  - The Flask application receives an HTTP POST request at /upload endpoint with a JSON payload containing the video URL.
  - It downloads the video from the provided URL and saves it locally in the uploads/ directory.
- Video Processing:
  - MoviePy Integration: Utilizes MoviePy for video processing tasks such as conversion to MP3 audio format, essential for subsequent transcription.
- Audio Transcription:
  - OpenAI's Whisper Model: The MP3 audio file undergoes transcription using OpenAI's Whisper model, chosen for its proficiency in converting audio to textual format through natural language processing (NLP) techniques.
- Text and JSON Generation:
  - Prompt Engineering: Before invoking Anthropic's Claude model, the transcribed text undergoes prompt engineering. This step involves crafting a structured prompt that directs the model towards generating coherent JSON data based on the processed transcription. The prompt is carefully designed to guide the model's response and ensure relevant and structured JSON output.
  - Anthropic's Claude Model: Integrated into the compute_engine.py module, Anthropic's Claude model is employed to generate structured JSON data from the processed transcription. This AI model is chosen for its capabilities in understanding and organizing textual data into meaningful and structured outputs suitable for content metadata.
- API Response and User Interaction:
  - Upon successful processing, the Flask application constructs a JSON response for the /upload request. This response includes the transcription text and a link to access the structured JSON data via /getjson/<filename>.

Decision Making Process

Real-time Matching: As users interact with the frontend service, the backend dynamically matches their content preferences against the JSON metadata associated with each video.
Algorithmic Decision: Using predefined algorithms, the service determines whether a video should be flagged for the user based on the alignment of flagged attributes with user preferences.
Reasoning and Transparency: Flagging decisions are transparently communicated through metadata, providing users with insights into why a video is flagged and ensuring accountability in content moderation.

Feedback Loop and Continuous Improvement

User Interaction: Flagged videos are visibly marked or highlighted in the frontend interface, empowering users to adjust their preferences or provide feedback on flagged content.
Iterative Refinement: Feedback from users drives iterative improvements in the flagging criteria and recommendation algorithms, enhancing the service's accuracy and relevance over time.

Security and Scalability

Secure Handling: API keys and sensitive data are securely managed using environment variables (dotenv), minimizing security risks and unauthorized access.
Scalable Architecture: The service architecture is designed to handle varying loads and scale efficiently, ensuring reliability and performance under high user demand.

Challenges Faced

One of the primary challenges was ensuring the accuracy and efficiency of the AI models in real-time content analysis. Balancing the sensitivity of content detection while avoiding false positives required iterative testing and optimization. Additionally, integrating the system seamlessly with TikTok's platform posed technical hurdles that we overcame through innovative engineering solutions.

MVP Features

Content Filtering: Real-time analysis and filtering of TikTok Reels based on user preferences.
User Alerts: Non-intrusive pop-ups to warn users about potentially distressing content.
Recommendation Engine: Updates recommendations based on user interactions and preferences.

Future Enhancements

Snooze Feature: Allow users to snooze content for a specified period.
Community Page: Enable users to view archives of blocked content, fostering transparency and trust in TikTok's content moderation practices.

SafeStream demonstrates the potential of AI to create a positive impact, making TikTok a leader in responsible content moderation and user experience.