Charlie

Inspiration

Charlie began in the autumn of 2024 as a personal experiment: could I build the digital secretary I had always wanted?

I did not want another chatbot that forgot everything when a conversation ended. I wanted an assistant that could gradually understand my work, the people around me, my ongoing projects and what had happened during my day—and then help turn that understanding into useful, structured action.

The original idea included voice, journaling, tasks, contact context, reminders and long-term memory. Several early versions were built and discarded as both my understanding of AI systems and the available models improved.

Over time, the idea became clearer:

Structured information should remain authoritative.
AI memory should be useful but rebuildable.
Important actions should be visible and reviewable.
The user should own the data and decide how autonomous the assistant may become.
The assistant should not be tied permanently to one user interface.

That became Charlie: an early working prototype of a local-first digital secretary and personal operations system. This is the n-th iteration, sixth? Seventh? This time with the renewed energy from wanting to use ChatGPT as the interface - something I assume will be possible in the near future.

What Charlie does

Charlie brings together several parts of everyday personal and professional organization:

Projects and tasks
Contacts and typed relationships
Journal and diary entries
AI-assisted journal review
Daily summaries
Persistent conversations
Long-term contextual memory
Model-facing tools through MCP
Reviewable AI-proposed actions

The intended experience is conversational.

A user talks or writes to Charlie about their day, current work, people they have spoken to and what should happen next. Charlie can then help organize that information into journal entries, contact context, projects and tasks.

For example, a conversation might contain:

I spoke with Anna about the reporting project. She will send the figures next week, and I need to prepare a draft before Friday.

Charlie should eventually be able to understand that as:

context connected to Anna;
a journal record of the conversation;
an update to the reporting project;
a task with an appropriate deadline;
information worth remembering in future conversations.

The structured database remains the source of truth. Long-term memory through Hindsight is treated as derived context rather than authoritative state.

How it works

Charlie is currently built as a provider-neutral modular monolith.

The React application uses a conventional FastAPI HTTP API. AI clients use a separately designed FastMCP interface. Both surfaces call the same application use cases, so business rules are not duplicated.

The current architecture includes:

Electron / React
        |
     FastAPI
        |
Application use cases
        |
PostgreSQL

For conversational interaction:

OpenRouter model
        |
Local chat orchestrator
        |
Curated in-process MCP tools
        |
Application use cases

For long-term memory:

Committed conversations
        |
Hindsight retain
        |
Rebuildable semantic memory
        |
Bounded recall into future conversations

Charlie’s PostgreSQL database remains authoritative for users, conversations, Projects, Tasks, Contacts, Journal records, permissions, versions and completed actions.

Hindsight stores only derived memory that can be deleted and rebuilt.

Safe and reviewable AI actions

One of the hardest questions has been how much authority an AI assistant should have.

Charlie currently allows the model to read current information through a tightly controlled set of MCP tools. When it wants to change authoritative data, it creates a structured proposal.

For example, the model can propose:

Create Task

Title: Finish the Build Week submission
Due: Tonight
Project: Charlie Desktop Preview

The user can inspect and approve or reject the proposal before it is executed.

The longer-term design will support user-configurable autonomy. Low-risk and reversible actions may be allowed automatically, while sensitive or destructive actions will continue to require review. A persistent approval queue is also planned so that conversations do not have to stop whenever several proposed changes need attention.

What I built during Build Week

Charlie existed before Build Week as an evolving personal project. The work completed during the competition period meaningfully extended it with two major vertical slices and the beginning of its desktop packaging.

Journal and AI review

I completed a substantial Journal system with:

immutable source material;
multiple entries per day;
manual and AI-assisted drafts;
strict structured-output validation;
human review before finalization;
links to Projects, Tasks and Contacts;
reviewable Task proposals;
daily summaries;
provenance and version controls;
OpenRouter and Amazon Bedrock provider adapters.

AI-generated material remains a proposal until it has been reviewed and accepted.

Local chat and persistent memory

I then implemented the first complete local conversational experience:

persistent conversation threads;
an OpenRouter-compatible model boundary;
actor-bound in-process MCP reads;
bounded and validated tool calls;
reviewable Task proposals;
Hindsight retain, recall and deletion;
owner-isolated memory banks;
memory rebuild operations;
safe degradation when memory is unavailable;
content-free telemetry;
strict provider timeouts and retry limits;
React chat routes and approval controls.

The implementation has been verified through backend, frontend, migration, contract, Compose, security and synthetic live-provider checks.

Desktop direction

For the submission, I also began packaging Charlie as a real Electron desktop application.

The intended desktop application contains the React interface, FastAPI backend, local PostgreSQL database and local Hindsight memory service. The goal is for users to install Charlie as a normal application without having to operate a collection of servers manually.

This desktop preview is still early, but it demonstrates the intended product direction: Charlie should feel like one coherent local assistant rather than a development stack.

Why MCP matters

I do not believe Charlie should permanently require users to interact through a dedicated Charlie interface.

The application exposes a carefully curated model-facing surface through MCP. This creates the possibility that ChatGPT, Claude or another conversational environment could eventually become the main user experience while Charlie provides the persistent personal layer underneath it.

In that model:

ChatGPT provides the conversation and intelligence;
Charlie provides durable user-owned context and structured operations;
MCP provides the shared interface between them.

Voice is the next major interaction step. The same orchestration path can accept a written message or a speech transcript, execute appropriate tools and produce a response suitable for text or speech. I've made several voice interaction apps before, using mainly OpenAI voice models combined with transcription and structured output to feed forms.

Challenges

The main challenge was not simply connecting an LLM to a database. It was deciding where the model’s authority should end.

Some of the most difficult areas were:

separating semantic memory from authoritative information;
preventing stale memory from overriding current Tasks or Contacts;
designing useful tools without exposing the entire application;
keeping model-generated writes structured and reviewable;
handling retries without creating duplicate effects;
maintaining owner isolation throughout HTTP, MCP and memory operations;
avoiding sensitive conversation content in logs and telemetry;
supporting multiple model providers without allowing provider details into the domain;
packaging several local services as one understandable desktop product.

Another challenge was resisting unnecessary abstraction. Charlie has gone through several architectures since 2024. During this implementation I deliberately chose concrete feature modules and narrow provider boundaries instead of building a generic “agent platform.”

What I learned

The most important lesson was that a useful personal assistant needs more than a powerful model.

It needs:

durable identity;
structured state;
explicit ownership;
reliable tools;
bounded memory;
reviewable side effects;
clear failure behavior;
a product experience that hides the underlying infrastructure.

I also learned that MCP is valuable not because every part of an application should use it, but because it creates a portable, model-oriented contract over the operations that an assistant should be allowed to perform.

FastAPI remains the right interface for the ordinary application. MCP is the right interface for models. Both can share the same application logic.

Finally, I learned how effective Codex can be as part of a disciplined engineering workflow. I used it not only to generate code, but to help research dependencies, develop specifications, inspect architectural boundaries, implement migrations, write tests, identify security problems and harden the resulting system.

What comes next

The immediate roadmap includes:

completing the Electron desktop package;
adding speech-to-text and text-to-speech;
introducing a persistent approval queue;
allowing configurable automatic approval for low-risk actions;
supporting user-added external MCP integrations;
improving local backup and recovery;
exploring a natural integration with ChatGPT;
eventually making Charlie usable from mobile devices.

The long-term goal is not an autonomous agent that silently controls a user’s information.

It is a trusted assistant that becomes more helpful over time while remaining understandable, inspectable and under the user’s control.

A personal note

I only discovered this competition on the afternoon before the deadline.

This is therefore less of a polished competition entry and more of an appreciation submission—and a glimpse of something I am genuinely excited about.

ChatGPT has had a remarkable effect on my life.

My background is in political science, education research and data analysis. I previously worked for a Swedish municipality as a researcher and analyst in its education department. Today, I work with AI insights and analytics at one of Sweden’s leading fintech companies.

The tools created by OpenAI played a meaningful part in that journey. They changed what I believed I could build and, honestly, what I believed I could do professionally - in fact, what I currently do.

Charlie is still early, but it is already far beyond what I would previously have thought possible for one person to build.

Thank you, sincerely, for making that possible.