Second Brain

A multi-agent system that reads, files, and synthesizes your notes automatically - so your second brain stays organized without you doing the organizing.

Comment

Inspiration

My notes have always been scattered: tasks in Notion, quick stuff in Apple Notes, a running pile of things I meant to come back to and never did. I was already migrating to Obsidian, and I realized a second brain is only as good as your first one. If you're not organized to begin with, an agent won't fix that, it just builds a prettier prison around the same mess.

So instead of automating the chaos, I built the organization first: one clear layer per kind of content, one writer per layer. Then I built an agent to run it.

What It Does

Second Brain Agent is a multi-agent pipeline that runs my Obsidian vault autonomously, so I never have to touch the upkeep myself.

Four agents, one job each:

  • Classifier — reads every new note the moment it lands and decides: task, project update, or knowledge
  • Filer — routes tasks straight into Tasks.md, no approval needed
  • Wiki-Synthesizer — rewrites the knowledge base as one clean current page, merging into an existing page if the topic's already there instead of appending endlessly
  • Brief-Generator — runs nightly, unattended, and writes me a real morning brief: project states, stale items, connections between notes, and the one thing my notes keep circling but I've been avoiding

The guardrail that matters most:

Tasks are low-risk, so they get filed immediately. Project status changes are not. Those are never applied silently, they get written as a proposal to Firestore and surfaced in the next brief for me to approve myself. Nothing rewrites Projects.md without me seeing it first.

How I Built It

Stack:

  • Gemini, via Vertex AI
  • Google ADK, orchestrating the four agents as a graph
  • Cloud Run, hosting the deployed service
  • Firestore, for operational state only
  • Cloud Storage, backing the vault itself
  • Pub/Sub + Cloud Scheduler, driving the two triggers
  • rclone, bridging the local Obsidian vault to the cloud

The two triggers:

A new note landing in the bucket fires a Cloud Storage → Pub/Sub → Cloud Run event chain, so the real-time pipeline runs without any polling. A separate Cloud Scheduler job drives the nightly brief, independent of the per-note path.

State, kept honest:

Firestore holds only derived, operational data, pending approvals, pipeline run logs, never a duplicate of the actual vault content. The bucket stays the single source of truth. rclone's sync direction is set per path to match a strict single-writer-per-layer rule, so I (the human) and the agent never write to the same file, which quietly solves the sync-conflict problem for free.

Challenges I Ran Into

  • The inbox isn't append-only. I edit and clear my capture file the same way I used to with a Notion inbox, so a Pub/Sub-message-ID or GCS-generation-based idempotency check would misfire on every edit. I redesigned it around a content-hash ledger instead, one mechanism that correctly handles new, unchanged, edited, and re-added notes.
  • A timezone bug hiding in the math. The server computed staleness in UTC while I'm in Central time, causing an off-by-one on task and project age. It only surfaced once a real threshold got exercised during eval, not in casual testing.
  • A real live-data incident, caught the same session. A routine eval run once briefly polluted my actual live Tasks.md. I fixed it immediately and then isolated eval infrastructure onto its own bucket, so it can't happen again.

Accomplishments That I'm Proud Of

  • The full loop is genuinely hands-off, end to end: I write a note into my real vault and, without touching anything else, it gets classified, filed or synthesized, and shows up correctly, or as a pending proposal if it's a project update.
  • The nightly brief runs unattended at 5am and has already surfaced a real approval queue from my actual life, not test data.
  • This isn't a demo I'll throw away after tonight, it's infrastructure I'm actually using.

What I Learned

The hardest part of an agentic system isn't the LLM calls, it's state management at the edges: idempotency against a file that isn't append-only, timezone correctness across local dev and Cloud Run, and IAM scoped tightly enough that a bug in the code can't cause damage a bug in the prompt already shouldn't allow. The single-writer-per-layer rule turned out to solve two problems at once, drift prevention and sync-conflict avoidance, for free.

What's Next

  • Switching from rclone to the Remotely Save Obsidian plugin, for in-app sync with no terminal step
  • Moving connections and cross-linking off the current approach, which leans on stuffing the whole vault into the LLM's context window each run, and onto a graph database (Cloud Spanner's graph support) or a RAG-based retrieval layer, so it keeps working as the vault grows past what fits in context
  • Adding the Sunday vault health audit pass, designed for but not yet built

Built With

  • cloud-scheduler
  • cloud-storage
  • firestore
  • gemini
  • google-adk
  • google-cloud-run
  • graph-workflow
  • multi-agent-system
  • obsidian
  • pub-sub
  • python
  • rclone
  • terraform
  • vertex-ai
Share this project:

Updates

Submission history