Inspiration

We live with constant mental noise. Ideas, reminders, emotions, inspirations arriving at random moments. Most tools force us to organize before we think. We wanted the opposite.

Cognitive Inbox is inspired by a simple belief: We remember the past and plan the future, so we can stay present.


What it does

Cognitive Inbox is a calm, multimodal personal AI assistant that lets users capture raw thoughts instantly, via text, voice, or images, without categorizing or deciding what to do next.

Behind the scenes, AI gently organizes inputs into meaningful groupings (ideas, tasks, reflections, wishlists), surfaces lightweight summaries, and offers optional assistance only when the user asks.

It’s a thinking space, not a productivity dashboard.


How we built it

We built Cognitive Inbox using Google Gemini’s multimodal and long context capabilities as the core intelligence layer.

Key components:

  • Multimodal input: text, voice (speech to text), and images
  • Gemini powered processing:
    • Transcription for voice
    • Image understanding (objects, text, context)
    • Intent detection and emotional tone analysis
    • Soft, probabilistic multi-label classification
  • Two-layer architecture:
    • Layer 1: Capture & Organize (always on, low pressure)
    • Layer 2: Assist & Execute (explicitly triggered)
  • Structured memo objects derived from raw inputs for grouping, summaries, and search

The system prioritizes human-centered interpretation over rigid schemas.


Challenges we ran into

  • Avoiding over-structuring
    Allowing ambiguity without losing usefulness
  • Emotional interpretation risk
    Ensuring reflections feel safe and non-invasive
  • Balancing intelligence with calmness
    Making the AI helpful without becoming noisy or pushy

Accomplishments that we're proud of

  • Thought capture in under 1 second, with zero required fields
  • Multimodal understanding using Gemini across text, voice, and images
  • Clear separation between memory and action
  • Gentle summaries that feel reflective, not judgmental
  • A product experience that feels quiet and non-demanding

What we learned

  • People don’t want to optimize thinking—they want to feel understood
  • The best AI assistants know when not to act
  • Soft, probabilistic organization feels more human than rigid rules
  • Emotional safety is as important as technical accuracy

What's next for Cognitive inbox

  • Long-term pattern detection (themes, cycles, recurring thoughts)
  • Memory with emotional context over time
  • User adjustable sensitivity for categorization
  • Calendar and timeline views
  • Cross device sync and deeper personalization

Cognitive Inbox aims to be a quiet companion, holding thoughts gently until users are ready to shape them.

Built With

  • ai-agent-design
  • browser-media-apis
  • fastapi
  • google-ai-studio
  • google-gemini-3
  • javascript
  • multimodal-ai
  • next.js
  • next.js-api-routes
  • prompt-engineering
  • react-native
  • restful
  • speech-to-text
  • sqlite
  • tailwind-css
  • typescript
  • uvicorn
  • vercel
Share this project:

Updates