live sell

Hold up a product on camera — Gemini AI detects it, pulls inventory, answers buyer questions live, and lets viewers buy in one click. Your live stream is now a storefront.

Comment

Inspiration

Live commerce is a $500B market in Asia (TikTok Shop, Taobao Live), but hosts still manually type product details, answer repetitive questions, answer repetitive questions, and lose buyers to friction.

We asked a simple question:
What if the camera itself was the sales engine?

A host holds up a dress — AI handles everything else.

What it does

LiveShop is a real-time AI agent that turns any live stream into a shoppable storefront.

See

Gemini Live API watches the host's camera and auto-detects products as they're shown.

Know

An ADK agent queries Firestore inventory to pull real prices, colors, sizes, and stock levels.

Talk

Viewers ask questions in chat like:
"does it come in red?"

The AI responds instantly using real inventory data.

Sell

Floating product cards appear on-stream with:

  • one-click buy
  • 10-minute reservation
  • checkout URL

All in real-time.

No manual listing.
No copy-pasting.
No missed sales.

How we built it

AI

  • Gemini 2.0 Flash Live for real-time video stream analysis
  • Product detection directly from camera frames

Agent

  • Google ADK for autonomous tool calling
  • The agent decides when to:
    • search inventory
    • check stock
    • reserve items

Database

  • Firestore
  • Real-time inventory
  • Atomic stock transactions
  • 10-minute reservation TTL

Workflow

  • Temporal.io
  • 8-state state machine managing the full stream lifecycle

Compute

  • Cloud Run
  • Runs the Temporal worker (Gemini + ADK processing)

Agent Platform

  • Rilo AgentEx (Kubernetes)
  • ACP server
  • WebSocket connections

Streaming

  • WebSocket pipeline

Host camera → JPEG frames every 2 seconds → AI detection → Product cards pushed to viewers

Challenges we ran into

Temporal gRPC routing

Exposing Temporal gRPC from Kubernetes to Cloud Run was difficult.

nginx ingress HTTP/2 framing broke Temporal’s gRPC protocol.

Solution:

  • TCP passthrough via nginx tcp-services-configmap
  • Bypassed L7 routing entirely

Google namespace conflicts

The uv package manager breaks google.* namespace when installing multiple google-* packages.

Fix:

Install dependencies in separate steps

pip install google-adk
pip install google-genai

Cloud Run health checks

Temporal worker does not expose HTTP, but Cloud Run requires a health endpoint.

Solution:

Minimal async health server:

asyncio.start_server(...)

It binds to $PORT before any Temporal connection attempts.

Self-signed TLS

Cloud Run connecting to an internal Kubernetes cluster with self-signed certificates required:

  • Monkey-patching the Temporal SDK TLS configuration

Accomplishments that we're proud of

End-to-end real-time pipeline

Camera frame → Product card in under 3 seconds

Zero manual input

The host never types:

  • product name
  • price
  • description

Production-grade inventory

Firestore atomic transactions prevent overselling.

Automatic reservation expiry handles abandoned carts.

Hybrid deployment

  • Cloud Run → scalable AI processing
  • Kubernetes → stateful agent orchestration

Connected across cluster boundaries.

ADK autonomous tool calling

The agent independently decides whether to:

  • search inventory
  • check stock
  • answer from context

What we learned

Gemini Live API is incredibly fast at visual understanding from raw JPEG frames.

No preprocessing needed.

Google ADK removes the need for manual orchestration between:

  • LLM
  • database
  • tools

The hardest part of AI agents isn't the AI.

It's the infrastructure plumbing:

  • gRPC routing
  • namespace packages
  • TLS
  • health checks

For hackathon streaming:

WebSocket frame ingestion

canvas.toBlob(...)
ws.send(blob)

is much simpler than WebRTC.

What's next for LiveShop

AI Voice Co-Host

Gemini Live audio output speaks alongside the human host, answering questions out loud so all viewers hear it.

Multi-language responses

Automatically detect viewer language and respond in:

  • Arabic
  • Hindi
  • English

Perfect for UAE market fit.

Shoppable replay

Recorded streams with timestamped product cards so viewers who missed the live can still buy.

Real-time scarcity engine

Dynamic overlays like:

  • "3 viewers looking at this"
  • "1 left in red"

Driven by Firestore real-time listeners.

Dynamic bundle suggestions

AI cross-sells products based on items shown in the same stream session.

Built With

  • cloud
  • cloudflare
  • docker
  • fastapi
  • gemini-2.0-flash
  • gemini-2.0-flash-live-api
  • genai
  • google
  • google-adk
  • google-adk-(agent-development-kit)
  • google-cloud-artifact-registry
  • google-cloud-firestore
  • google-cloud-run
  • google-genai
  • helm
  • kubernetes
  • next.js-15
  • nginx-ingress
  • pydantic
  • python-3.12
  • react
  • secret
  • temporal.io
  • temporalio
  • typescript
  • webrtc
  • websocket
Share this project:

Updates