Speaknote

SpeakNote turns your voice into organized reminders — just speak to the built-in AI assistant, and let it handle the details, dates, and follow-ups while you stay focused on what matters.

Comment

Inspiration

To reduce the daily friction of managing reminders. Typing out tasks on a small phone screen, setting dates manually, choosing categories — it all takes some time. The idea was to build a mobile-first reminder app where a voice command to an AI assistant named "Lenny" could handle the heavy lifting. Just speak, and the reminder gets created. Built with the Replit Mobile App Builder, SpeakNote was designed to feel like a conversation with a capable assistant, while staying firmly in the utility lane — not a chatbot, but a task tool.

What it does

SpeakNote is a voice-first AI reminder application. Users tap a microphone button, speak naturally — "Remind me to call the dentist tomorrow at 3pm" — and an AI agent named Lenny processes the command, asks any clarifying questions, and creates the reminder.

Key capabilities:

  • Voice-to-reminder pipeline: Speak a command, Lenny using Google Gemini 2.5 Flash and creates the reminder
  • Multi-turn AI conversations: If Lenny doesn't have enough information, it asks follow-up questions — "What time works best?" or "How urgent is this?"
  • Full reminder management: Create, edit, complete, archive, delete, and share reminders. Subtasks, categories, priority levels, recurring schedules, and calendar views are all included.
  • Smart suggestions: AI-generated insights and tips based on the user's active reminders.
  • Dark mode: Full dark mode support across every page, toggled from Settings and persisted locally.

How it was built

SpeakNote was built entirely on Replit using the Mobile App Builder, from database design through deployment. The process followed a structured approach:

  1. Data model first: The schema was designed in shared/schema.ts using Drizzle ORM with PostgreSQL — six tables covering users, reminders, categories, subtasks, templates, and notifications.
  2. Backend API layer: Express.js routes were built for authentication, full CRUD operations for reminders and categories, and then AI-powered endpoints.
  3. AI integration: Google Gemini 2.5 Flash was integrated for voice command processing, multi-turn reminder creation (the "Manager" agent), smart suggestions, and general AI commands. The TTS endpoint uses the gemini-2.5-flash-preview-tts model.
  4. Frontend pages: Twelve React pages were built using Vite, TanStack Query, wouter for routing, and shadcn/ui components styled with Tailwind CSS.
  5. Voice interaction flow: The VoiceCommand page was built with the Web Speech Recognition API for input, a multi-turn conversation UI with chat bubbles and progress indicators, and a dual TTS system (Gemini primary, browser fallback).

The entire build was done on Replit's platform, leveraging its built-in PostgreSQL database and deployment pipeline.

Challenges

  • TTS reliability: Browser Speech Synthesis availability varies wildly across devices and browsers. The solution was implementing Gemini's TTS model as the primary voice, with the browser API as a graceful fallback.
  • Category display consistency: The app initially showed raw category UUIDs in various views. Fixing this required threading category data through every page that displays reminders — Home, Calendar, AllReminders, and ReminderDetails.
  • Dark mode coverage: Ensuring dark mode looked right across twelve different pages with various component styles required systematic CSS class additions across the entire frontend.
  • Voice recognition edge cases: Handling the Web Speech Recognition API's inconsistencies — premature stops, no-speech errors, browser compatibility — while maintaining a smooth user experience.

Accomplishments

  • Built a fully functional voice-to-reminder pipeline using AI, from speech input through structured data extraction to database storage.
  • Implemented a multi-turn AI conversation system that intelligently identifies missing reminder fields and guides users through completion.
  • Integrated Gemini TTS as a reliable voice output system that works even when browser speech synthesis fails.
  • Shipped a complete mobile-first application with twelve pages, full authentication, and polished dark mode — all built on Replit.

What I learnt

  • Building voice interfaces requires multiple fallback layers — both for speech recognition input and text-to-speech output
  • Multi-turn AI conversations in a utility context need careful guardrails to stay task-focused and avoid drifting into general chat territory.
  • Replit's integrated development environment — with built-in database and deployment — significantly accelerates full-stack development.
  • Mobile-first design decisions made early (like touch-friendly tap-to-record and large interaction targets) paid off in a much smoother end-user experience.

What's next for SpeakNote

  • Push notifications for upcoming reminders using service workers
  • Widget-style quick-add from the home screen
  • Integration with external calendars (Google Calendar, Apple Calendar)

Built With

Share this project:

Updates