CineFact AI

Autonomous workstation powered by Gemini & Parallel Search API: turns raw video into fact-checked 35-45s viral shorts with speech-aligned subtitles and dynamic trust badges.

Comment

Inspiration

Short-form video dominates modern social media distribution, yet distilling hour-long podcasts, technical keynotes, and executive interviews into high-retention social clips remains a tedious manual process. Even worse, algorithmic feeds are saturated with unverified claims and hallucinations that spread unchecked.

We built CineFact AI to reinvent the post-production workflow into an autonomous, agentic cinematic workstation. Powered by Gemini 3.8 Flash multimodal intelligence and the Parallel Search API for real-time web verification, CineFact AI transforms raw footage into speech-aligned, fact-checked, 35–45 second social clips, complete with synchronized subtitles, dynamic trust badges, and multi-angle editorial cuts ready for instant distribution.

What it does

CineFact AI functions as an end-to-end autonomous video intelligence workstation:

  • Multimodal Asset Ingestion: Accepts direct video uploads (.mp4, .webm, .mov) via a resilient chunked upload pipeline and streams YouTube URLs directly into the analysis engine.
  • Autonomous 35–45s "Golden Window" Discovery: Evaluates vocal inflections, narrative momentum, and visual pacing with Gemini 3.8 Flash to extract high-retention highlight windows strictly calibrated between 35 and 44.5 seconds (targeting an optimal ~40s sweet spot).
  • Frame-Accurate Speech Boundary Alignment: Employs an intelligent boundary-snapping algorithm that aligns cuts with natural sentence completions (., !, ?) and vocal pauses, completely eliminating clipped syllables or mid-sentence chops.
  • Director’s Multi-Cut Suite (4 Editorial Variations): Autonomously generates four distinct narrative perspectives from a single asset:
    • Cut A (Viral Hook): High-energy opening hook (~40s) designed for top-of-funnel engagement.
    • Cut B (Deep-Dive Evidence): Core technical lore and data points (~38s) for informative depth.
    • Cut C (Punchline & Climax): The concluding realization or actionable takeaway (~40s).
    • Cut D (Key Moments Digest): A cohesive 3-part montage (Hook + Evidence + Resolution) stitched into an unbroken 35–45s story.
  • Live Claim Verification via Parallel Search API: Automatically isolates testable claims from the transcript, executes real-time web search queries against https://api.parallel.ai/v1/search, and computes confidence scores, domain authority ratings, and verifiable source citations.
  • Studio Clearance Dossier: Generates an auditable compliance and factual clearance dossier displaying aggregated trust scores, claim breakdowns, and clickable verification references.
  • Synchronized Dual-Player Workspace: Features an interactive timeline scrubber and waveform editor with simultaneous dual-player monitoring, comparing the original footage side-by-side with the rendered highlight short.
  • Multi-Format Server-Side FFmpeg Exporter: Renders production-ready MP4 files across 9:16 (Vertical Shorts/Reels/TikTok), 1:1 (Square), 4:5 (Social Feed), and 16:9 (Landscape YouTube), featuring blurred ambient canvas framing and optional hard-burned subtitles and fact-check badges.

How we built it

  • Frontend Workstation: Built with React 18, TypeScript, and Tailwind CSS, featuring dual-player playback controls, responsive waveform scrubbing, dynamic aspect ratio switchers, and inline subtitle customization.
  • Multimodal Agent Core: Powered by the @google/genai SDK and Gemini 3.8 Flash with structured JSON schemas and an automated model fallback hierarchy (gemini-3.8-flash > gemini-3.7-flash > gemini-3.6-flash) to guarantee zero downtime during upstream traffic spikes.
  • Real-Time Fact Verification: Integrated the Parallel Search API to perform live semantic search across web infrastructure, returning structured evidence dossiers, credibility scores, and source URLs.
  • Deterministic Media Processing: Engineered a Node.js and Express backend with native server-side FFmpeg pipelines for frame-accurate slicing, multi-segment audio/video concatenation, ambient background generation, and subtitle burn-in.
  • Resilient Chunked Uploader: Implemented a binary chunked upload mechanism that reliably handles large video files without browser memory exhaustion or request timeout limits.

Challenges we ran into

  • Mid-Sentence Audio Clipping: Early automated cuts strictly adhered to fixed timestamps, frequently cutting speakers off mid-word. We resolved this by building a dedicated speech-boundary alignment engine that analyzes transcript timestamps and snaps cut boundaries to natural sentence pauses.
  • Multi-Segment Audio/Video Parity in Montages: Concatenating discontinuous video moments into Cut D introduced stream ordering mismatches and filtergraph link crashes. We engineered an FFmpeg pipeline with strict video and audio pad interleaving ([v0][a0][v1][a1]...concat=n=N:v=1:a=1), PTS re-basing, and audio timestamp normalization to ensure seamless transitions.
  • Browser CORS & Canvas Recording Limitations: Relying on client-side canvas capture resulted in dropped frames and muted audio across various browsers. We replaced client-side recording entirely with an asynchronous server-side FFmpeg rendering engine that returns native, broadcast-standard MP4s.
  • Dev/Prod Environment Lifecycle on Cloud Run: Injecting environment secrets inside Cloud Run dev sandboxes initially triggered false production mode flags. We resolved this by decoupling container service detection from Vite development middleware, ensuring seamless live updates and hot module reloading.

Accomplishments that we're proud of

  • End-to-End Autonomous Pipeline: Going from raw, unedited video footage to four distinct, professionally cut, and fact-verified social shorts in under 30 seconds.
  • Speech-Boundary Precision: Achieving natural, broadcast-grade dialogue pacing that respects the speaker’s natural cadence without manual editing intervention.
  • Transparent Information Trust: Bringing verifiable truth to social video through the Parallel Search API and the Studio Clearance Dossier, providing creators and viewers with real-time confidence scores and primary-source citations.
  • Robust Dual-Player Architecture: Building a desktop-grade video workspace that allows creators to audit, trim, and inspect editorial cuts with sub-second responsiveness.

What we learned

  • Autonomous media tools require a hybrid architecture: combining probabilistic multimodal intelligence (Gemini 3.8 Flash) for creative narrative understanding with deterministic media engineering (FFmpeg and audio boundary snapping) for flawless execution.
  • Grounding generative video summaries with live search verification (Parallel Search API) is the single most effective defense against misinformation in automated content production.

What's next for CineFact AI

  • AI Face-Tracking & Smart Auto-Reframing: Integrating facial landmark tracking to dynamically pan and keep active speakers centered during multi-person debates and interviews.
  • Direct Social Publishing: One-click OAuth publishing webhooks to export finalized clips directly to TikTok, YouTube Shorts, and Instagram Reels.
  • Live Broadcast Fact-Checking: Extending the pipeline into real-time WebRTC streams for live political debates, earnings calls, and conferences.

Built With

  • gemini-3.8-flash
  • google-ai-studio
  • google-cloud-run
  • parallel-search-api
  • tailwind-css
Share this project:

Updates

Submission history