DalilNotecard

The AI speech co-pilot for presenters, teachers & tour guides. Gemini AI automatically checks off your notecard points as you talk hands-free.

Comment

DalilNotecard — Real-Time Hands-Free AI Speech Co-Pilot

💡 Inspiration

Public speakers, educators, professors, and tour guides face a common dilemma: how to stay structured without breaking eye contact with their audience.

Traditional handheld index cards, paper notes, or smartphone screens distract from live human connection. Existing presentation software requires manual clicking or tapping. We set out to build an invisible, hands-free co-pilot that listens to the speaker's voice in real-time, understands natural speech and context, and automatically checks off covered bullet points on screen as they speak.

🚀 What It Does

DalilNotecard transforms physical speech notes into an active digital co-pilot:

  1. Continuous Speech Recognition: Captures ambient voice audio with browser wake-lock support for extended hands-free lectures.
  2. Hybrid Semantic Matching Engine: Combines instant 0ms local keyword matching with the Google Gemini 3.5 Flash-Lite API (with automatic fallback to Gemini 3.6 Flash) to detect paraphrased ideas, homophones, speech-to-text typos, and conversational rephrasing.
  3. Real-Time Quota & SaaS Management: Integrates serverless usage tracking in Google Cloud Firestore, offering a generous 1-hour free tier with automatic RevenueCat subscription webhook upgrades.

🛠️ How We Built It

DalilNotecard is built on a high-performance, serverless Google Cloud Platform (GCP) architecture:

  • Frontend: React 18, TypeScript, Vite, Web Speech API, and PWA offline support.
  • Backend & Serverless API (GCP):
    • Firebase Cloud Functions (v2): Secure Node.js proxy endpoint (checkCoveredPoints) calling the @google/generative-ai SDK with an adaptive model cascade (gemini-3.5-flash-lite → gemini-3.6-flash → gemini-3.5-flash).
    • Firebase Secret Manager: Secures Gemini API keys and RevenueCat webhook authorization tokens.
    • Cloud Firestore: Real-time user profile sync, usage analytics (aiSecondsUsed), and IP-hashed guest tier limits.
    • Firebase Hosting: Global CDN deployment delivering sub-second initial page loads.

📐 Technical Architecture & Latency Math

To deliver a smooth co-pilot experience, total round-trip latency must remain under 1.5 seconds:

$$\text{Latency}{\text{total}} = \text{STT}{\text{stream}} + \text{Network}{\text{RTT}} + T{\text{Gemini 3.5 Flash-Lite}} < 1.5\text{s}$$

By utilizing Gemini 3.5 Flash-Lite for rapid inferencing, average AI processing time is reduced to $\sim 300\text{--}450\text{ms}$.

🧠 AI-Native Operations & Unit Economics

DalilNotecard was designed from day one to operate autonomously with AI in production:

  • Every active presentation executes an adaptive background AI loop every $2.5\text{s}\text{--}12\text{s}$, evaluating the evolving transcript against remaining unchecked points.
  • Profitable Unit Economics: For a typical 45-minute continuous lecture (~540 API calls at 3,000 avg input tokens/call), unit cost is calculated as:

$$\text{Cost}_{\text{lecture}} \approx 540 \times \$0.000231 = \$0.125\text{ USD} \quad (\sim \$0.166\text{ / hour})$$

For an active speaker performing 20 hours of lectures per month ($\sim \$3.32/\text{month}$ API expense), a premium subscription of $\$9.99/\text{month}$ yields a strong gross operating margin:

$$\text{Gross Margin} = \frac{\$9.99 - \$3.32}{\$9.99} \times 100\% \approx 66.8\%$$

⚠️ Challenges We Overcame

  1. Handling Speech-to-Text Noise: Speech-to-text engines often transcribe homophones or minor errors (e.g., "foreclosure story" instead of "folklore story"). We engineered custom Gemini prompt instructions to perform lenient phonetic and contextual matching.
  2. Preventing API Quota Exhaustion: Built a dual-stage pipeline where local matching handles $\sim 70\%$ of exact phrase matches, invoking Gemini only for unchecked or paraphrased items.
  3. Mobile Screen Management: Implemented the Screen Wake Lock API to prevent mobile displays from dimming during long presentations.

🎓 What We Learned

Deploying Gemini 3.5 Flash-Lite (backed by Gemini 3.6 Flash) in a continuous audio loop proved that modern LLMs are fast and cost-effective enough to act as invisible, real-time co-pilots in live physical environments without cumbersome local hardware requirements.

🔮 What's Next for DalilNotecard

  • Multi-Language Speech Matching: Expanding Gemini prompts to support Arabic, Spanish, French, and Japanese lectures.
  • Automated Slide Deck Sync: Integrating Google Slides API to automatically advance presentation slides as bullet points are checked off.
  • AI Post-Lecture Summaries: Generating automatic Q&A recaps and student study notes after each presentation.

Built With

Share this project:

Updates