ctxindex

Inspiration

This didn't start as a hackathon idea. I built it because I was annoyed with the setup I already had.

My Hermes agent used a collection of provider-specific command-line tools, including gog. It could do useful things, but I couldn't get multiple accounts working reliably. Search was split between tools, each provider supported different things, and sometimes the agent would simply miss something important.

I started building a better version for myself. While doing that, I realized it shouldn't become another giant tool with every integration hard-coded into it. A smaller core with Extensions made much more sense and could also be useful to people using agents other than Hermes.

That became ctxindex, pronounced “context index.”

What it does

ctxindex gives agents one command-line interface for searching and retrieving context from email, calendars, files, and other sources.

It currently works with Gmail, Outlook, Google Calendar, Microsoft Calendar, local directories, and anything added through an Extension.

You can connect multiple accounts, group Sources into Realms such as personal or work, search across them, and retrieve the complete result using a stable ctx:// reference.

The important part is that agents don't need a special integration. If an agent can run shell commands and read structured output, it can use ctxindex. I use it with Hermes, but the same commands work from Codex, Claude Code, or another shell-capable agent.

Everything stays local. Google, Microsoft, and the original files remain the source of truth. ctxindex keeps a local index and cache that can be deleted and rebuilt.

I also deliberately limited what it can change. It can create or update an email Draft, but it cannot send email.

In the video, I ask Hermes to find important GitHub emails from the previous two days. Most of my real uses involve private mail, calendars, and files, so I couldn't show very much of the actual data publicly.

How I built it

I built ctxindex as a TypeScript and Bun monorepo. It has a local daemon, SQLite storage, a CLI, Google and Microsoft support, an Extension SDK, tests, and a public documentation website.

Extensions can add new types of context, providers, adapters, authentication, actions, documentation, and assets. They can be installed from npm, Git, a local directory, or a Catalog. There is also an official Catalog that acts as the marketplace.

I built most of the project with GPT-5.6. I started in Pi and later moved the work to Codex.

In Codex, I kept one very large session open as a meta-session. I planned and made decisions with the root agent while smaller subagents worked on specific implementation, testing, research, and review tasks. That worked surprisingly well. It let us work on several independent parts without losing the overall direction of the project.

Codex's browser and computer tools were also genuinely useful. Setting up real OAuth applications meant spending a lot of time inside Google Cloud Console and Microsoft Entra. Those interfaces are huge, and having the agent help navigate and check them saved a lot of time.

ctxindex had an early foundation before Build Week. The work completed during the week is visible in the public commit history and dated agent sessions. I also want to be clear that the earlier Pi work was done in Pi, not Codex, even though both used GPT-5.6.

Challenges I ran into

The hardest part wasn't calling the Gmail or Microsoft APIs. It was deciding what the basic concepts should be so that the same system still made sense across different providers and multiple accounts.

OAuth was painful. Google and Microsoft have different terminology, different consoles, and different restrictions. A setup that works for one personal account might not work for an organization account.

Extensions also took several attempts. I wanted them to be powerful without turning the core into a collection of special cases.

I spent a lot of time on smaller details that agents depend on: stable references, predictable errors, exit codes, output formats, cancellation, packaging, and making sure the installed CLI behaves like the version in the repository.

There was also the simple challenge of time. This became a much larger project than I expected to finish in one week.

Accomplishments that I'm proud of

I'm proud that ctxindex is a real tool and not just a demo built for the video.

The CLI is published on npm. The repository is public and MIT licensed. The website, documentation, Extension examples, no-credentials demo, and official Catalog are all available publicly.

I'm also proud of the codebase. A lot of work went into keeping responsibilities separate, writing down the important decisions, and testing the behavior at several levels. The CLI, daemon, core, provider adapters, and Extension SDK have clear boundaries instead of everything being tied together.

That probably isn't the most exciting thing to show in a short video, but it matters to me because I plan to keep using and developing it after the hackathon.

What I learned

The biggest thing I learned is that giving an agent access to personal context isn't just a search problem.

The agent needs to know where something came from, which account it belongs to, whether the result is complete, how to retrieve it again, and what it is allowed to change. Without that, even a very capable model ends up guessing.

I also learned that subagents work much better when each one gets a small job with clear ownership. Letting many agents make broad changes at the same time creates a mess. Using the main session to plan and review while focused subagents handled separate pieces worked much better.

What's next for ctxindex

First, I'm going to keep using it with Hermes and fix the problems I find in real use.

After that, I want to add more providers and context types, improve search across Sources, expand the official Catalog, and make creating and publishing Extensions easier.

I also want the first-time setup to become much simpler, especially OAuth. That is currently the least pleasant part of using the project.

There are many directions ctxindex could go, but I want to keep the basic idea simple: agents should have one reliable way to reach the context you already own.

Built With

• codex
• gpt5.6