Viral Arbiter AI: The E-commerce Trend Scout

💡 Inspiration

Finding the right product before the market gets saturated. I wanted to build a "scout" that never sleeps—an agent that analyzes social signals to find winning products automatically.

🏗️ How I built it

The project is built on a robust Big Data pipeline:

• Data Ingestion: Python-based scrapers and API integrations for TikTok, Pinterest, and Google Trends.
• Processing: Used a Lakehouse architecture approach to handle both structured trend data and unstructured social media content.
• AI Intelligence: Integrated with the Prompt Opinion platform as an MCP Server to allow other agents to query real-time product viability scores.
• ML Models: Comparison of Random Forest and XGBoost to classify "winning" vs. "failing" products based on historical growth curves.

🧠 What I learned

I deepened my expertise in orchestrating workflows (Airflow) and learned how to transform raw social sentiment into a structured data warehouse that an AI agent can actually understand and act upon.

🚀 Challenges I faced

The biggest challenge was the latency of social signals. Dealing with real-time TikTok trends requires efficient data containerization and fast indexing, which I solved by optimizing the data lakehouse structure.

Built With

• airflow
• docker
• kubernetes.-machine-learning:-xgboost
• languages:-python-(3.11+)
• manage
• minio-(s3-compatible-lakehouse-storage-for-unstructured-social-data).-apis:-google-trends-api
• or
• orchestration:
• pinterest
• prefect
• prompt-opinion-a2a-sdk.-storage-&-db:-postgresql-(metadata)
• random-forest-(for-predictive-classification).-platform-protocols:-mcp-(model-context-protocol)
• scheduled
• scikit-learn
• scraping
• sql.-big-data-&-processing:-apache-spark-(for-trend-analysis)
• tiktok-research-api
• trends.