TrueLift AI

Real-time retail AI that catches bad promotions before they drain profits. Confluent streams + Gemini AI = instant detection and smart recommendations.

Comment

Inspiration

Every year, retailers lose $8.7 billion to promotional cannibalization—the hidden phenomenon where discounted products steal sales from full-price inventory instead of generating true incremental revenue. Traditional analytics catch this problem days or weeks later, long after the damage is already done. Marketing teams review spreadsheets while margins evaporate in real-time.

We asked ourselves: "In a world where promotions launch instantly and shopping behavior shifts by the minute, why does it take days to detect when promotions are destroying more value than they create?"

Consider these real scenarios:

  • A "Buy One Get One" promotion on laundry detergent steals 60% of sales from the premium line
  • A flash sale on running shoes cannibalizes the new seasonal collection's launch
  • Deep discounts train loyal customers to wait for sales, destroying long-term pricing power
  • Regional promotions cause cross-channel arbitrage, hemorrhaging margin across the organization

The combination of Confluent's real-time data streaming and Google's Gemini AI gave us the perfect toolkit to tackle this challenge. We envisioned an intelligent system that doesn't just report problems—it explains them, recommends solutions, and learns from every action taken.

That question became TrueLift AI—a Real-Time Promotional Intelligence Platform that detects cannibalization the moment it happens and responds with AI-powered strategies.

What It Does

TrueLift AI is the "Radar System for Retail Promotions". Just like air traffic control detects deviations in real-time, TrueLift AI instantly identifies when promotions deviate from expected performance and guides managers to corrective action.

Core Capabilities

Feature Description
Real-Time Detection ML models compare actual vs predicted sales in 1-minute aggregation windows
Liquidity Scoring AI-powered severity classification (Low/Medium/High) based on loss percentage
Gemini Pro Insights Human-readable explanations of why cannibalization occurs, with data-driven recommendations
One-Click Actions Stop promotions, adjust pricing, or target specific segments instantly
Feedback Loop Measures effectiveness of every action, creating a self-improving system
Live Dashboard WebSocket-powered real-time updates for immediate visibility
Demo Simulator One-click burst generation to showcase the entire pipeline end-to-end

The Revenue Loss Formula

$$\text{Loss} = (\text{Predicted Sales} - \text{Actual Sales}) \times \text{Promo Price}$$

When loss exceeds configurable thresholds, alerts are generated with severity levels based on the loss percentage:

  • High (≥40%): Immediate action required—promotion is destroying value
  • Medium (20-40%): Significant cannibalization—consider adjustments
  • Low (<20%): Monitor closely—potential optimization opportunity

Dynamic Action Recommendations

Action When to Use Expected Impact
Stop Promotion Deep cannibalization (>40% loss) 70-95% loss reduction
Adjust Price Moderate loss (20-40%), test elasticity 30-50% loss reduction
Target Segments Light loss (<20%), improve targeting 15-35% loss reduction

How We Built It

Tech Stack

We chose technologies optimized for real-time processing and AI-powered decision making:

Layer Technology Purpose
Frontend Next.js 14 + React 18 Real-time dashboard with TypeScript and WebSockets
Backend FastAPI + Python 3.11 REST API and WebSocket server
ML/AI Vertex AI + Gemini Pro Cannibalization detection and AI reasoning
Streaming Confluent Cloud Kafka Real-time event streaming
Processing Apache Flink Stream aggregations
Database Cloud SQL (PostgreSQL) Product catalog and operational data
Cache Redis Fast data access and strategy caching
Deployment GCP Cloud Run Serverless microservices with auto-scaling

Architecture Philosophy

We designed TrueLift AI with an event-driven, microservices architecture that scales each component independently:

┌─────────────────────────────────────────────────────────────────────────────┐
│                         PHASE 1: REAL-TIME DETECTION                        │
│                                                                              │
│   Virtual Shoppers ──► Kafka [sales-events] ──► ML Detector (1-min windows) │
│                                                                              │
│   • Simulates realistic shopping behavior                                    │
│   • Compares actual sales vs predicted baseline                             │
│   • Generates cannibalization alerts when loss exceeds threshold            │
└─────────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                         PHASE 2: AI STRATEGY GENERATION                      │
│                                                                              │
│   Kafka [cannibalization-alerts] ──► Gemini Agent ──► AI Recommendations    │
│                                                                              │
│   • Context-aware prompt engineering with product and alert data            │
│   • Strategy caching with rate limiting to manage API costs                 │
│   • Confidence scoring for recommendation quality                           │
└─────────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                       PHASE 3: DASHBOARD & ACTION LOOP                       │
│                                                                              │
│   Backend API ──► WebSocket ──► Dashboard ──► User Actions ──► Feedback     │
│                                                                              │
│   • Real-time updates via WebSocket connections                              │
│   • One-click action execution (stop, adjust, target)                        │
│   • Feedback loop measures action effectiveness                              │
└─────────────────────────────────────────────────────────────────────────────┘

Key Implementation Details

  1. Event-Driven Processing: Sales events flow through Kafka, ensuring decoupled, scalable processing. Each service consumes from specific topics and publishes to downstream topics, creating a resilient pipeline.

  2. Intelligent Prompt Engineering: The Gemini Agent constructs context-rich prompts including product pricing, discount percentages, predicted vs actual sales, and historical alert patterns for each promotion.

  3. Strategy Caching: To manage Gemini API costs and handle burst scenarios, we implemented Redis-based caching with configurable TTL. Repeated alerts for the same promotion reuse cached strategies during the rate-limit window.

  4. WebSocket Broadcasting: When new alerts or strategies are generated, they're immediately broadcast to all connected dashboard clients via an internal broadcast endpoint, ensuring sub-second visibility.

  5. Feedback Loop Integration: When users take actions, the feedback-loop service monitors subsequent sales patterns to calculate effectiveness scores, creating a closed-loop learning system.

Challenges We Ran Into

1. Real-Time Consistency Across Distributed Services

Challenge: Ensuring alerts, strategies, and feedback stay synchronized across 6 microservices (virtual-shoppers, ml-detector, gemini-agent, feedback-loop, backend, frontend) required careful coordination.

Solution: We designed idempotent Kafka consumers with proper offset management and used Redis as a shared state layer for strategy caching and cross-service coordination. Each alert carries a unique alert_id that threads through the entire pipeline.

2. Cold Start Latency in Serverless Environment

Challenge: Cloud Run's serverless model scales services to zero, causing 5-15 second delays on first requests after idle periods.

Solution: We optimized container sizes, implemented health check endpoints in background threads, and set appropriate timeout configurations. For the demo experience, we documented expected cold start behavior and optimized the Docker images.

3. Gemini API Rate Limits During Burst Scenarios

Challenge: During demo bursts with multiple simultaneous alerts, Gemini API rate limits could cause delays or failures.

Solution: We implemented a multi-tier strategy:

  • Redis-based caching with 15-minute TTL for identical promotions
  • Rate limiting to prevent duplicate strategy generation within time windows
  • Fallback strategies with reasonable defaults when API is unavailable
  • Max poll records configuration in Kafka consumers to throttle processing

4. Database Schema Evolution

Challenge: As we added features (burst tracking, feedback metrics, pricing metadata), we needed to migrate schemas without losing existing data or breaking running services.

Solution: We designed additive schema changes with nullable columns and backward-compatible queries. The initialization script uses IF NOT EXISTS patterns, and we added troubleshooting documentation for local development resets.

5. WebSocket Reliability and Reconnection

Challenge: Maintaining persistent WebSocket connections for real-time updates across browser refreshes, network interruptions, and service restarts.

Solution: We implemented automatic reconnection logic with exponential backoff on the frontend, connection status indicators in the UI, and graceful handling of disconnection states. The dashboard shows "Live" or "Offline" status so users always know their connection state.

Accomplishments That We're Proud Of

End-to-End Real-Time Pipeline

From event generation to AI strategy display in under 3 seconds. This is the full path: shopper event → Kafka → ML detection → alert creation → Kafka → Gemini analysis → strategy storage → WebSocket broadcast → dashboard update.

Production-Ready Architecture

Despite being a hackathon project, TrueLift AI has:

  • 6 independently deployable microservices on GCP Cloud Run
  • Proper secret management via GCP Secret Manager
  • Complete audit trail with timestamped alerts, strategies, and actions
  • Scalable event streaming via Confluent Cloud Kafka
  • Cost optimization with min-instances=0 and capped max instances

Novel AI Application

The Gemini Strategy Agent is the first system we know of that generates context-aware promotional cannibalization strategies in real-time. It doesn't just detect problems—it explains why they're happening and recommends what to do with confidence scores.

Closed-Loop Learning System

The Feedback Loop service monitors the impact of every action, calculating effectiveness scores based on before/after sales patterns. This creates a self-improving system where the measured impact of past actions informs future recommendations.

One-Click Demo Experience

The Demo Simulator became our killer feature. Click "Start Demo" and watch:

  1. Virtual shoppers generate a burst of 6 event batches
  2. ML detector identifies cannibalization patterns
  3. Alerts appear with DEMO tags showing burst metadata
  4. Gemini generates AI strategies in real-time
  5. Take an action and watch the feedback loop calculate impact

What We Learned

Technical Learnings

  1. Event-Driven Architecture Shines for Real-Time AI: Kafka's decoupling of producers and consumers allowed us to independently scale the ML detector, Gemini agent, and feedback loop based on their unique resource requirements.

  2. Confluent Cloud Makes Kafka Accessible: We went from zero Kafka experience to a production-ready streaming pipeline in days. The hosted Flink SQL and topic management eliminated operational overhead.

  3. Gemini Pro Excels at Contextual Business Recommendations: With proper prompt engineering including product context, pricing data, and sales metrics, Gemini generated nuanced, actionable strategies that felt like they came from an experienced retail analyst.

  4. Caching is Essential for AI Workloads: Without strategy caching, repeated alerts for the same promotion would hammer the Gemini API and slow down the pipeline. Redis caching reduced API calls by 80%+ during burst scenarios.

  5. GCP's Ecosystem Works Seamlessly: Cloud Run + Cloud SQL + Secret Manager + Artifact Registry work together with minimal configuration friction. The VPC connector pattern for private database access is elegant.

Product Learnings

  1. Demo-Ability Drives Understanding: The Chaos Simulator was originally a testing tool, but it became the most effective way to explain the product. Seeing the entire pipeline animate in real-time tells the story better than any slide deck.

  2. Trust Through Transparency: By showing confidence scores, explaining why strategies were recommended, and tracking action effectiveness, we transformed a black-box AI into a trusted decision support tool.

  3. Edge Cases Define Quality: The difference between a prototype and a product is handling the edge cases—cold starts, rate limits, reconnection logic, stale data indicators. These details make users trust the system.

Team Learnings

  1. Domain Research Pays Off: Understanding real retail cannibalization scenarios helped us design realistic simulation patterns and meaningful demo experiences.

  2. Microservices Enable Parallel Development: Each team member could work on different services (frontend, backend, ML detector, Gemini agent) simultaneously without blocking each other.

  3. Start with the Happy Path, Then Harden: We built the end-to-end demo flow first, then systematically hardened each component with error handling, retries, and fallbacks.

What's Next for TrueLift AI

Immediate Roadmap

Priority Feature Description
High Confluent Flink Integration Use Flink SQL for sophisticated windowed aggregations beyond 1-minute windows
High Predictive Models Train on historical feedback data to predict which strategies work best for different products
High RAG Enhancement Integrate with product catalogs and competitor data for richer AI context
Medium Multi-Store Expansion Support regional and store-level analysis with hierarchical alerts
Medium Automated Actions Allow pre-approved actions to execute without human intervention
Medium Mobile Alerts Push notifications via Twilio for urgent cannibalization events
Future A/B Testing Framework Built-in promotional experimentation with statistical significance
Future External Data Integration Weather, events, and competitive intelligence for contextual predictions

Technical Improvements

  • Real-Time WebSockets: Replace polling with server push for sub-second updates
  • Redis Caching Optimization: Hierarchical caching for product, promotion, and strategy data
  • PostgreSQL Partitioning: Time-based partitioning for historical alert data at scale
  • Load Testing: Support 10,000+ concurrent users during peak promotional periods
  • CI/CD Pipeline: Automated testing and rolling deployments via Cloud Build

Vision

TrueLift AI is positioned to become the operational intelligence layer for retail promotions. We envision:

  1. CPG Partnership: Pre-integrated with Nielsen and IRI data for manufacturer-retailer collaboration
  2. Marketing Platform Integration: Native connectors to Salesforce Marketing Cloud, Adobe Experience Platform
  3. Retailer Suite: Complete promotional lifecycle management from planning to post-analysis
  4. Enterprise Dashboard: Multi-tenant platform for retail chains with centralized insights

"In 2025, promotions launch in seconds and shopping behavior shifts by the minute. When tools that analyze promotions take days, you're not analyzing—you're post-morteming."

TrueLift AI: Detect. Understand. Act. Learn.

Built for the Google Cloud + Confluent Hackathon

Detecting promotional cannibalization before it costs you money!

Built With

Share this project:

Updates