Inspiration
Meetings are where teams make decisions, assign work, expose disagreements, and create the context needed to act. Yet most meeting tools either upload the original audio or flatten everything into a disposable transcript full of “Speaker 1” and “Speaker 2.”
We built MASK to become a private, durable memory layer on the Mac—and a bridge between human conversation and Codex. It remembers not only what was said, but who said it, across meetings. That speaker-aware context can stay fully local or be handed to a user-controlled Codex task when the user wants to move from memory to action.
What it does
MASK is a native macOS meeting-memory app whose core audio pipeline runs locally on Apple Silicon.
- MOSS transcribes and performs context-aware speaker diarization together.
- Voice embeddings connect local speaker labels across chunks and recognize the same person in later recordings.
- Manual corrections always take precedence over automatic identity matches.
- Scenes organize recurring meetings and learn their participants from the recordings placed inside them.
- Live mode records the microphone, system audio, or a drift-compensated mix of both.
- An interactive 3D PCA view shows where each person's voiceprints sit in embedding space.
- Users can generate meeting minutes, a TL;DR, or a per-speaker viewpoint analysis.
- The interface is available in English and Traditional Chinese.
From private memory to Codex context
MASK is useful by itself, but its larger possibility is as a local context layer that extends what Codex can do.
A transcript alone is weak context. MASK can provide speaker-aware, cross-meeting memory: the people involved, their recurring viewpoints, the decisions they reached, unresolved questions, and the action items assigned to each person. Instead of asking Codex to begin from a pasted wall of text, the user can open a task with structured context already prepared.
The shipped integration demonstrates the complete loop:
- MASK freezes the selected transcript, verified speaker names, template, and fingerprint.
- It opens the official
codex://threads/newdeep link with a pre-filled task. - Codex reads the immutable snapshot from MASK's localhost server.
- Codex reports concise stages—reading, outlining, comparing speakers, writing—back into the native UI.
- The final Markdown is posted back to MASK and stored beside the recording.
- The one-time capability expires immediately.
Today this loop powers meeting minutes, TL;DRs, and per-speaker viewpoint analysis. The same capability-based bridge creates a path to task templates that move beyond summarization: turn decisions into implementation plans, inspect the relevant repository, implement an agreed action item, run tests, draft a follow-up document, or prepare the next meeting from unresolved questions. MASK supplies durable human context; Codex supplies the ability to act on it. The user remains in control because the prompt is reviewed before sending and every localhost capability is scoped to one job.
Two explicit privacy modes
The built-in option uses a 2-bit Ternary-Bonsai-27B model on MLX, so the full workflow—including transcript and summary—stays offline.
The optional Codex mode avoids the 8.5 GB summary-model download. The audio file still never leaves the Mac; only the structured transcript is intentionally handled under the user's OpenAI/Codex plan.
These modes are an explicit user choice because they have different privacy contracts. MASK does not silently fall back from one to the other.
Security and human control
The Codex callback is not a broad unauthenticated webhook. Every job receives an unguessable, short-lived capability held only in memory. It can access only that job's frozen snapshot and is invalidated on completion or failure.
The callback prompt forbids returning hidden reasoning or chain-of-thought. If somebody edits the transcript while Codex is working, the frozen fingerprint causes the returned result to be marked stale instead of silently claiming it matches the new content.
Speaker identity follows the same principle: voiceprints provide evidence, but they never replace MOSS's contextual turn labels or override a manual correction.
How we built it
MASK is an SPM multi-module codebase:
- MaskData: GRDB schema, records, and repositories
- MaskCore: pure domain logic, pipelines, identity, summaries, and the Codex coordinator
- MaskEngine: MLX/CoreML engines, MOSS, VAD, Smart Turn, and live Core Audio capture
- MaskServer: FlyingFox REST, SSE, WebSocket, and Codex callback endpoints
- MaskCLI: headless transcription and server commands
- Mask App: a thin SwiftUI shell
The SwiftUI app and HTTP handlers both call the same MaskServices facade. Speaker information has three distinct layers: MOSS local labels, recording-level clusters, and cross-recording people.
How Codex and GPT-5.6 helped
Codex and GPT-5.6 were development collaborators throughout Build Week. We used them to trace the multi-module architecture, diagnose a silent Core Audio tap from the allow-list versus exclusion semantics, design the one-time capability protocol from first principles, implement coordinated domain/server/SwiftUI changes, add English localization, and write focused unit and HTTP integration tests.
Codex is therefore both a tool that helped build MASK and a user-facing runtime: a bridge that lets private meeting memory become useful context for an agent capable of doing the follow-on work.
Challenges and accomplishments
The hardest boundary was combining contextual diarization with persistent identity without letting imperfect embeddings overwrite a human decision. Separating local labels, clusters, and people made that contract explicit.
Another subtle failure was a Core Audio process-tap initializer that accepted an empty process allow-list and produced valid-looking but silent buffers. Switching to the global tap initializer with an empty exclusion list fixed the actual capture semantics.
For the Codex bridge, transcript snapshotting, progress callbacks, scoped authorization, lifecycle cleanup, and honest privacy copy all had to work as one product contract.
Validation
The final Release build succeeds, and 80 tests across 23 suites pass. Coverage includes database migrations, diarization invariants, persistent identity across recordings, summary map-reduce, API/OpenAPI completeness, and the Codex capability/progress/completion lifecycle.
What's next
- Add action-oriented Codex task templates for repository work, follow-ups, and issue creation.
- Add notarized distribution and automatic updates.
- Add opt-in encrypted export/import for person and voiceprint libraries.
- Expand languages and acoustic testing while keeping manual correction authoritative.
Built With
- core-audio
- fluidaudio
- gpt-5.6
- grdb
- mlx
- moss
- openai-codex
- swift
- swiftui