The problem: Videos sell products better than any other format — but today, buying what you see in a video still takes a detour. Viewers have to leave the video, follow a link, land on a separate page, and only then check out. Every extra step loses buyers. What ClipPay does: ClipPay turns a raw video into a shoppable experience in three steps: 1. Upload a video. ClipPay transcribes it with timestamps (faster-whisper). 2. Products are extracted automatically. A Gemini model reads the transcript and pulls out buyable products with names and prices — no manual tagging. 3. One tap buys, inside the player. Products appear as cards beside the video. Tap one, and PayPal checkout opens. Payment completes and the seller sees the order on their dashboard. How it is different: TikTok, YouTube, and Instagram shopping are locked inside their own platforms, and products still have to be tagged by hand. ClipPay is different: drop in any video, it transcribes and extracts products automatically, and one tap buys — no manual tagging, no platform lock-in. Why this is agentic commerce: ClipPay is not a dashboard with a Buy button bolted on. It is an AI agent that watches a video, understands what is being sold, builds the storefront, and carries the buyer through PayPal checkout — with no human tagging in the loop. The AI does the selling work; the human just taps to confirm. Built with: FastAPI + SQLite backend; faster-whisper for transcription; Gemini for product extraction (structured JSON output); PayPal Orders API v2 (sandbox) for checkout and capture. Verified end to end (PayPal Sandbox, 2026-10-24): Real transcription + Gemini extraction run: 3 products extracted from a sample sales-pitch video (Handmade Ceramic Mug $20.00, Walnut Cutting Board $45.00, Organic Cotton Tote Bag $12.00). Full buyer flow run in PayPal Sandbox: $1.00 test order created → buyer approved → capture COMPLETED. Order ID: 5BL51414A7794864L. Capture ID: 86W38762EW7659830. Honest notes: The demo video is a staged-data demonstration: transcription, extraction, and the sandbox order reuse results verified live on 2026-10-24; it is not a live re-run during filming. The in-player checkout shown in the video uses the clearly labelled demo/mock checkout path; the real PayPal Sandbox payment is presented separately with its real order and capture IDs above. Product timestamps are coarse (the sample's 22 seconds transcribe as a single segment), so products are tappable throughout playback rather than triggered per second.

Built With

Share this project:

Updates

Submission history