Nina.io

Inspiration

Ads in live streams today fail to match the dynamic and unpredictable nature of stream content. We’re building a real-time ad-generation engine that analyzes live video and chat data to generate context-aware ads that adapt to what’s happening on screen—voiced and personalized through the streamer’s own tone and personality. By syncing ads with the live context and audience sentiment, we transform advertising from an interruption into an engaging, native part of the stream.

What it does

To solve this issue, we built Nina.io, a platform that analyzes the real-time context of feeds and delivers ads tailored to the specific moment, content, and audience. Nina.io captures segments of the live feed and processes them through advanced AI to analyze visual content, audio, chat interactions, and viewer sentiment. Using this analysis, Nina.io generates a contextually relevant ad concept through an LLM, which produces a custom script. The ad is then delivered back to viewers, creating a seamless, personalized advertising experience that feels native to the broadcast rather than intrusive

How we built it

• Reka AI: Handled the video stream analysis, processing live feeds in real-time to extract visual context and understand what's happening on screen.

• Custom Chat Analysis Agent: Built to monitor and analyze live chat interactions, capturing viewer sentiment and conversation flow to understand audience engagement in real-time.

• Google Gemini: Using Gemini's Live API, we built a conversational agent capable of generating contextually relevant ad scripts and pitching advertisements in an interactive manner that adapts to user feedback.

• Fish Audio: Cloned the streamer's voice using audio samples, enabling ads to be delivered in their actual tone and style for a natural, native feel.

• Postman Flow: Integrated the voice cloning functionality into our backend, coordinating between ad script generation and voice synthesis.

• Replit: Served as our frontend platform, allowing us to execute our end-to-end flow with a clean, functional UI.

Challenges we ran into

The biggest challenge was building a pipeline that could maintain low latency while performing sophisticated real-time analysis. When you're processing live video feeds, analyzing chat sentiment, generating contextual ad scripts, and cloning voices, every millisecond counts. We had to carefully optimize each step of our pipeline to keep response times fast enough that the ads felt natural and timely, not delayed and awkward.

Integration was another major hurdle. We were working with multiple AI services, each with their own APIs, rate limits, and quirks. Getting Reka AI, Gemini, and Fish Audio to work together seamlessly while managing WebSocket connections and coordinating between our MongoDB and PostgreSQL databases required constant debugging and architectural refinements.

Balancing quality with speed was a constant tradeoff. We wanted high-fidelity video analysis and natural-sounding voice clones, but we also needed the system to respond in near real-time. Finding that sweet spot took a lot of iteration.

Accomplishments that we're proud of

Integration! Trying to integrate across our stack proved to be challenging! Balancing low latency with high fidelity analysis was our biggest engineering challenge. However, through careful optimizations within our pipeline, we were able to preserve response time while providing intelligent analysis in near real time.

What we learned

Real-time systems require a completely different mindset than traditional web applications. You can't afford to wait for sequential API calls or perform heavy processing in a blocking manner. We learned to think in terms of pipelines, parallel processing, and asynchronous workflows.

Integration is harder than it looks on paper. Even when APIs are well-documented, combining multiple services into a cohesive system surfaces edge cases and performance bottlenecks you don't anticipate. We got much better at debugging distributed systems and understanding how different services interact under load.

Context is everything in AI applications. The quality of our ad generation improved dramatically when we refined how we extracted and summarized context from video and chat. Generic prompts produced generic results, but when we fed the models rich, specific context about what was happening in the stream, the output became genuinely engaging.

What's next for Nina.io

AI-generated video ads! While today’s AI video generation models still face latency and quality constraints that make true real-time ad insertion impractical, we see enormous potential in this space. Video is one of the most engaging formats for user impact — so although we’re currently prioritizing real-time performance and stability, we plan to introduce AI-personalized video ads as generation models become fast and controllable enough for live delivery.