Envoy

• 

  Thumbnail

• 

  Landing page

• 

  Brief upload

• 

• 

  Call

• 

  De-brief

Inspiration

As a student balancing part-time work, important classes, and online meetings, I often face schedule clashes. Most AI note-takers only summarize a call after it ends. I wanted to build something more useful: an AI that can represent you in the meeting, share approved updates, and make recommendations within the boundaries you set and more importantly feel like you, make recommendations that you approve and gives you a summary after the call for your approval.

What it does

Envoy is a disclosed AI meeting delegate. It represents one named person and works only within that person's approved authority.

Before the meeting, the user adds:

1. The person's name and role
2. The person's mandate and limits
3. The meeting purpose
4. The required result
5. Approved source materials

During the meeting, Envoy listens quietly. It responds only when a participant says “Envoy” or types @Envoy. Envoy gives the answer in chat and can speak the same answer. Each answer includes a source line when source information is available. Envoy identifies itself once. It does not pretend to be the person that it represents. If a request exceeds its authority, Envoy stops the action. It then asks the represented person for approval. After the meeting, Envoy creates a brief. The brief contains: a. Decisions b. Action items c. Source information d. Approval requests e. Approval results

The current demonstration takes approximately one minute. The meeting continues while participants remain active.

How we built it

We built the interface with React, TypeScript, and Vite. A Node.js server manages the meeting sessions and OpenAI requests. The OpenAI Responses API creates answers from approved information. Envoy also supports OpenAI File Search for approved knowledge. The Web Speech API manages browser speech input and speech output. One event pipeline processes voice requests and @Envoy chat requests. A wake-name gate filters ordinary speech. Duplicate checks prevent repeated answers. A risk check sends restricted decisions to the human approval flow. We used Codex during design, implementation, testing, debugging, and deployment. Codex helped us remove old project concepts, improve the user flow, add tests, and resolve deployment failures.

Challenges we faced

The main challenge was to make Envoy act like a representative instead of a general assistant. We had to solve these problems:

1. Slow answers
2. Duplicate speech events
3. Incomplete speech input
4. Unwanted voice activation
5. Repeated disclosure messages
6. Answers that exceeded the person's authority
7. Output loop
8. We also had to make the complete product clear in a short demonstration.

What Existed Before Build Week

Before OpenAI Build Week, this repository contained an incomplete RepAI meeting-assistance prototype. The initial concept and some Microsoft Teams experiments already existed, but the project was not a complete, reliable, or deployed Envoy product.

What We Added During Build Week

During the submission period, I used Codex with GPT-5.6 to substantially extend and complete the project. Codex audited the inherited codebase and helped transform the earlier prototype into Envoy, a disclosed AI meeting delegate. It implemented and refined the pre-meeting, live-meeting, and post-meeting journey. It added addressed @Envoy chat, wake-name voice activation, duplicate-event protection, one-time disclosure, approved-material grounding, human approval controls, meeting briefs, and automated tests. Codex also diagnosed problems that had prevented earlier versions from working reliably. It reduced response latency, fixed repeated responses and cut-off speech, completed the interface redesign, repaired the Docker and Railway deployment, and verified the final application through tests, production builds, and browser checks. GPT-5.6 supported the engineering and product work through Codex. It also powers Envoy’s grounded meeting responses through the OpenAI Responses API when the live API configuration is enabled. The original idea existed before Build Week. The working Envoy product, its safety model, completed experience, deployment, and submission package were produced through the substantial Build Week extension documented in the dated commit history and primary Codex session.

Accomplishments

We completed these parts: a. A complete pre-meeting, meeting, and post-meeting flow b. One response path for voice and chat c. Answers that use approved source information d. Visible source information e. Human approval for restricted decisions f. A wake-name control for spoken requests g. A clear and consistent Envoy identity h. A live application that judges can use

What we learned

An AI representative needs more than a good language model. It needs clear authority, approved information, stable turn control, and human approval. Disclosure must also be clear. However, repeated disclosure makes the product feel unnatural. Envoy gives this information once and keeps it visible in the interface.

What is next

We plan to add: a. Production connectors for Zoom, Microsoft Teams, and Slack b. More document formats c. Organization approval policies d. Faster real-time voice e. Consent-based voice profiles f. Local processing options for sensitive information h. View the Envoy source code

Build Week Evidence

Built With

• docker
• node.js
• openai-api
• openai-file-search
• openai-responses-api
• react
• rest-api
• typescript
• vite
• vitest
• web-speech-api