Minute

MINUTE turns online meetings/classes into a structured workspace powered by multimodal Gemini 3: live recap, cited Q&A from session/project memory, and post-session minutes plus tailored outputs.

Comment

Inspiration

We wanted the clarity of NotebookLM-style, source-grounded Q&A, but for the messy reality of live meetings and online classes. We also took cues from how people naturally organize work in the Google ecosystem: documents living in folders, sessions tied to shared materials, and knowledge accumulating over time. At the same time, tools like Otter, Fathom, and Fireflies showed how valuable automatic transcripts and summaries can be—but we kept running into a gap between a good recap and a truly usable workspace.

In practice, every call becomes a trail of transcripts, slides, links, and follow-ups—yet the context is scattered across apps and tabs. MINUTE started from a simple idea: treat each session as a first-class workspace where evidence, highlights, and next steps are automatically captured, organized, and easy to scale

How we built it

We built a Next.js, Python-based web app with a session hub and an in-session side panel. Audio is streamed to an ASR layer to produce timestamped transcript segments. In parallel, we sample video frames and detect slide changes using lightweight perceptual hashing, then send key frames to Gemini 3 for multimodal understanding. A window builder merges transcript excerpts and visual events into context windows, which Gemini 3 uses to generate recap updates. For Q&A, we implemented tiered retrieval: session memory first, then uploaded documents (hybrid keyword + vector search with filters), followed by optional web search behind a human approval gate.

Challenges we ran into

The hardest part was time alignment. Audio and video arrive on different clocks, so we standardized everything to a server timebase and tuned alignment rules around slide-change anchors. Latency was a constant trade-off: we constrained vision calls to high-signal frames, used short rolling windows with overlap, and cached intermediate results to keep interactions responsive.

Infrastructure and cost trade-offs were equally real. Multimodal processing can get expensive fast, especially if you sample too aggressively or retry too often. We addressed this by packaging the pipeline into clearly bounded stages, adding rate limits and backpressure, and using progressive fidelity: cheap heuristics first (hash/SSIM, low-rate sampling), then Gemini 3 only when a frame is likely to add value. We ran repeated evaluations across different sampling rates, window sizes, and thresholds, comparing speed, stability, and recap quality until we found a balanced configuration that performs well on typical sessions.

Finally, we had to prevent confident, ungrounded answers. That pushed us to enforce citations by default, add evidence coverage checks, and provide a safe fallback when sources are insufficient.

Accomplishments that we're proud of

  • A working multimodal timeline that meaningfully connects what was said to what was shown
  • Tiered, citation-first Q&A that stays grounded in session and uploaded sources
  • A practical session workspace UX that feels like a real product, not just a transcript viewer
  • A cost-aware pipeline that remains responsive without sacrificing usefulness

What we learned

We learned that building a reliable assistant is largely a workflow and product design problem. The model matters, but the user experience depends on how you structure context, surface evidence, and guide decisions. Treating each session as a workspace—timeline, transcript, sources, Q&A, and outputs in one place—dramatically improves trust and usability. We also learned that small orchestration choices (when to summarize, how to cite, what to cache, when to ask for approval) shape perceived intelligence more than raw generation quality. Good defaults, clear boundaries, and explainable outputs are what make the assistant feel dependable in real work.

What's next for MINUTE

Through this competition, MINUTE aims to find opportunities for real-world product validation, proving multimodal capabilities across live meetings and online classes. Future expansion includes production deployment with multi-tenant architecture, industry-specific customizations (finance compliance reports, academic quizzes), mobile companion apps, and API marketplace integrations, scaling from prototype to enterprise SaaS while maintaining citation-first reliability for teams and organizations worldwide.

Built With

Share this project:

Updates