GPT Intent

New product for OpenAI suite. Agentic messenger and open messenger protocol. "Why wait for user to prompt? Recognize intent and be agentic."

Comment

Intro

Inspiration

First, we wanted to give each person an agent and let agents talk to each other. We thought what if we can reconsider modern communication through messengers and email from the ground up. From the first principles.

But we faced the biggest of all startup problems: we had an idea, but saw very little use cases for it to be applicable. So we pivoted, and tried to understand what would be a more suitable place to introduce agents - that's how GPT Intent was born.

What it does

GPT Intent is a proposal for the first-party OpenAI agentic messenger. Messenger where you can talk with friends, family and other people, while having an agent by your side that can:

  • Generate hints based on your conversation history by doing proactive research before you even enter the conversation
  • Proactively propose agentic actions
  • Draft responses for you to explicitly pre-populate

Idea was much wider in a form of summaries, routines, ad-hoc tagging of an agent as third participant in the conversation itself, etc. But we tried to limit our POC to the ones described above.

At the submission time our Electron app can do the following:

  • User is able to choose among three providers
    • iMessages (Messages.app on macOS) via open-source iMessage Kit SDK
    • Telegram via official TDLib
    • Demo provider that seeds initial chat data with imaginary contacts and responds using Codex SDK
  • For inference application requires logging in with ChatGPT account and utilizes Codex SDK and app server
  • Upon connecting and landing in the main UI the application starts to parse conversations, generate hints using (configurable) ChatGPT model, propose follow up actions and complete those using plugins natively installed into main ChatGPT app

How we built it

Over the course of the week we churned over 3 generations of the app. Starting with initial iMessage-only "sidecar", pivoting into its own full messenger experience app, introducing Telegram and Demo provider specifically for evaluation purposes. We transitioned from ad-hoc generated design and renderer to utilizing industry standard libraries such as shadcn/ui, Tanstack set for list virtualization, routing and state management. Codex was used across the journey, utilizing long-running /goals (with a record of close to 3 hours for one of the pivots), computer use for agentic testing, "grilling" us on requirements, running discovery into unfamiliar areas (like how would the official Telegram lib built in C++ be integrated). We submit only a single session from /feedback, but actually we also used the recently released Entire platform to sync sessions from Codex alongside code produced in commits https://entire.io/gh/AndrewNikolin/GPTIntent. These sessions are also available on GitHub repo in entire/checkpoints/v1 branch and as refs. We think, total number of Codex threads is 100+. Thanks, Tibo, btw 😅, it wouldn’t be possible otherwise.

Challenges we ran into

Perhaps if we had a refined idea from the very beginning, and didn't have to pivot couple times before settling on the final product idea, the overall architecture and implementation would be more polished. Redesigning the frontend from scratch also didn’t help 😅. Tuning the system prompt, getting familiar with Codex SDK and app server also happened just-in-time, and the Codex itself helped a lot with discovery and integration. It also came up with a set of evals to adjust system prompt to set expectations, so generated action proposals and hints make more sense to the user We accepted that the current implementation isn't ideal from technical standpoint, since each pivot introduced technical debt and we were scrambling to implement the best vision we had, considering a hard deadline and the need to balance working on project in free time and background while fulfilling responsibilities on our full-time jobs.

Accomplishments that we're proud of

Considering all said, we are happy that the app actually provides a glimpse (with rough edges) into an idea of what the AI-first messenger app could be. We gained a lot of experience in building desktop applications in Electron, integrating Codex app server and SDK for inference, overcoming head-on collisions with pivots generated by long-running goals and genuinely having fun while building this project 6 time zones apart.

What we learned

  • Building full-blown agentic messenger in ~5 days (2-3 days for brainstorming) is challenging, even with Sol and Codex 😅
  • Most of our effort, time and struggles went into product vision, definition and UX brainstorming. Knowing what we know now, we’d say the chance of full-blown agentic messenger complete will be at least twice as high.
  • Be flexible. We had to pivot from the original idea of agent-to-agent communication - it wasn’t easy to accept at first, but it was the right call.
  • When life gives you resets - use them to the max!

What's next for GPT Intent

The hackathon is over, but for us - it is just the beginning. While the submitted version code and branch have to be frozen, we are already considering following up with all lessons learned, and starting to build GPT Intent v2 on a separate branch. This time giving it more thought in terms of architecture, technical debt and being less constrained in time. Our hope is to, at least, create a polished version of the application with Telegram at its core and publish it for global audience to evaluate, criticize and see how it evolves.

Ultimately, we’d want to work on this idea as part of OpenAI, not necessarily to win the hackathon or prove our idea is the best among others. Meaning that GPT Intent can be it’s own messenger, without the need to support Telegram or iMessage integrations. We see our Build Week submission not as a regular project, more like a proposal to you guys at OpenAI to see the vision and maybe work on it together in the future as first-party product. Of course, building it with grounded approach and not in the rush of a 1-week sprint.

Also, as mentioned in the presentation, we envision GPT Intent to also introduce open protocol communication standard (or use existing one), so that GPT Intent doesn’t become another walled garden (like Telegram or WhatsApp who are not open to communicate with other messengers).

We see the future where your ChatGPT Assistant is a single entity that stays with you wherever you are: messenger, work, browsers, real-life conversations (via GPT Puck), etc. It has the whole context of everything you do and remembers things just like a real person Assistant would. Completely synchronized across all the surfaces.

… at least until Neuralink makes us communicate with thoughts :-)

Built With

Share this project:

Updates