ONE

• 

• 

• 

  Make it YOU

Inspiration

Voice AI demos can feel magical. Then someone interrupts, stays silent, speaks over background noise, reaches voicemail, or loses their connection, and that magic can disappear.

After years of helping organizations turn APIs and SDKs into dependable customer experiences, I became fascinated by the gap between an impressive voice demo and a system people can genuinely trust. Building voice AI meant navigating fragmented providers, hidden events, incompatible interfaces, unpredictable costs, sensitive credentials, and failures that were difficult to reproduce.

I started ONE as a local learning lab where I could see the entire system working: authentication, payloads, transcripts, realtime events, latency, interruptions, recovery behavior, and the connection between an audio interaction and its intended human outcome.

As I explored more providers, a larger problem became clear. Creators are often asked to choose a platform before they have enough evidence to know whether it fits their needs. Provider comparisons tend to flatten everything into leaderboards, even though the best combination depends on language, latency, privacy, accessibility, geography, budget, and the experience being created.

ONE grew from a personal voice AI lab into a larger mission: give humans and agents an open place to understand the ecosystem, test it honestly, and assemble the combination that works best for them without surrendering control to a single vendor.

What it does

ONE is an open, provider-neutral lab for learning, building, testing, and improving voice AI.

The lab makes normally hidden system behavior visible, including:

• Realtime conversation states and event timelines
• Speech-to-text, reasoning, text-to-speech, and transport layers
• Interruptions, crosstalk, silence, latency, voicemail, and reconnect behavior
• Controlled failures, fallback paths, and recovery decisions
• Provider capabilities and implementation status
• Structured evaluation evidence
• Authentication and payload construction
• Explicit execution and provider-spend controls
• Reusable voice-agent architecture patterns

For the WebMCP experience, ONE exposes a focused workflow through four structured tools:

1. list_voice_scenarios
2. configure_voice_replay
3. start_deterministic_replay
4. get_replay_evidence

A person can ask their agent to test how a voice system handles an interruption or lost connection. The agent can discover the available scenarios, configure a deterministic replay, request execution, and retrieve the resulting evidence without guessing where to click.

The browser remains the shared workspace. Configuration changes appear visibly in ONE, consequential actions require confirmation, the test unfolds through the interface, and the resulting timeline and scorecard remain available for human review.

The agent handles repetitive discovery, configuration, and evidence gathering. The person retains intent, judgment, approval, and the final decision.

How we built it

ONE began as a local React-based voice lab and evolved into a modular system with provider adapters, realtime inspection tools, deterministic scenarios, evidence capture, and an extensive browser regression suite.

The most important architectural decision was creating one capability layer shared by the human interface and WebMCP. The agent does not operate a hidden copy of ONE or maintain a separate version of its state. WebMCP invokes the same capabilities, updates the same interface, and produces the same evidence that a person sees.

The collaboration loop works like this:

1. The person defines the goal and constraints.
2. The agent discovers an appropriate test.
3. The agent configures the scenario.
4. ONE makes the proposed configuration visible.
5. The person approves execution when required.
6. The replay runs inside the shared interface.
7. The agent retrieves and helps interpret the evidence.
8. The person decides what to investigate next.

I built deterministic local replay so reliability failures can be reproduced without requiring a live provider call or spending API credits. Live, simulated, experimental, and proposed capabilities are labeled separately so ONE never implies that something is operational when it is not.

Provider credentials remain server-side, while execution gates prevent a browser agent from silently initiating billable activity.

Challenges we ran into

The hardest challenge was not teaching an agent how to operate ONE. It was deciding what the agent should be allowed to do, what must remain under human control, and how every action could be inspected afterward.

Realtime voice is also inherently difficult to test. Network conditions, timing, microphones, background noise, provider behavior, and human speech can all change the result. Deterministic replay became the bridge between realistic failure scenarios and repeatable evidence.

Another challenge was translating a complex laboratory into a small set of meaningful WebMCP tools. Exposing every button would only recreate the interface as an API. I instead designed tools around complete intentions: discover a scenario, configure it, run it, and inspect what happened.

ONE also began as a substantial local project rather than a challenge-sized prototype. Preparing it for the web required disciplined scope, deployment hardening, browser validation, security boundaries, and careful separation between what is truly implemented and what remains part of the roadmap.

Accomplishments that we're proud of

ONE grew from a provider-specific learning experiment into a provider-neutral system for studying the complete voice AI stack.

I am especially proud of:

• Building deterministic reliability scenarios for interruption, crosstalk, silence, latency, reconnects, and other real-world conditions
• Creating a zero-provider-spend path for testing system behavior safely
• Making agent execution visible instead of allowing agents to act silently
• Preserving explicit human confirmation before consequential or billable actions
• Separating live, adapter-backed, experimental, simulated, and proposed capabilities
• Producing structured timelines and evidence instead of relying only on subjective impressions
• Building a shared capability layer that serves both the human interface and WebMCP
• Growing an extensive automated regression suite around a fast-moving experimental project
• Building this as an independent creator while keeping the architecture open to an entire ecosystem

The accomplishment I value most is that ONE does not treat human control as friction. Human judgment is part of the architecture.

What we learned

The biggest lesson is that voice quality is not a single model score. It is an end-to-end property of the system, including timing, turn-taking, transport, recovery, observability, permissions, cost, and human expectations.

I also learned that provider neutrality requires more than displaying several logos. It requires consistent capability descriptions, honest implementation states, comparable evidence, provenance, and the freedom to combine services rather than accept an entire bundled stack.

Agent-native design is not simply automation. An effective agent needs structured capabilities, but a trustworthy agent also needs bounded authority, shared state, visible history, and clear moments of human approval.

WebMCP creates that relationship inside the web experience. The agent does not replace the interface or move the work into an invisible backend. The person, page, and agent remain together.

Finally, I learned that hosted access and local sovereignty are not opposing ideas. The web can support discovery, education, and collaboration, while local execution can preserve privacy, resilience, portability, and user control.

What's next for ONE

Almost everything.

The long-term vision is for ONE to include every meaningful provider across the voice AI ecosystem: speech recognition, speech generation, speech-to-speech models, language models, telephony, orchestration, evaluation, translation, accessibility, observability, and supporting infrastructure.

The goal is not to identify one universal winner. It is to let any human or agent assemble the right system for a particular need and budget.

Someone might choose one provider for transcription, another for reasoning, another for voice, another for telephone transport, and a separate evaluation layer to verify the result. Another creator might prioritize local processing, a specific language, low latency, accessibility, regional data handling, or the lowest sustainable cost. ONE should help both of them construct an evidence-backed stack.

Agents will help people navigate this growing ecosystem by translating human requirements into testable configurations. They will compare current capabilities, identify missing evidence, run approved experiments, explain tradeoffs, and help assemble portable blueprints. People will remain responsible for the goals, values, constraints, and final choices.

The roadmap includes:

• More provider adapters and community-contributed integrations
• Comparable pricing, latency, language, privacy, and capability information
• Evidence freshness and provenance for every recommendation
• Human and agent-generated reliability scenarios
• Provider-neutral benchmarks based on reproducible tests
• Live inbound and outbound telephony blueprints
• Multilingual and accessibility-focused evaluation
• Local nodes for private or offline voice workflows
• Exportable architectures that creators can take elsewhere
• Signed evidence and auditable experiment histories
• Feedback loops that improve recommendations as more tests are completed
• WebMCP capabilities across the entire lab
• A complementary headless MCP interface for approved workflows beyond the browser

Most importantly, ONE is intended to move beyond an enterprise-only ecosystem. A student, independent creator, local business, researcher, nonprofit, community organization, or global company should all be able to enter the same lab and find a path appropriate to their resources.

ONE should become a living map of voice AI rather than another closed platform inside it.

The vision is enormous, but the method is concrete: add one provider, one scenario, one transparent comparison, and one piece of verified evidence at a time.

The goal is to make voice AI easier to explore without making it harder to understand, and more powerful without taking agency away from the people it is meant to serve.

Built With

• accessibility
• agents
• chatgpt
• codex
• deepgram
• human-in-the-loop
• mcp
• node.js
• opensource
• playwright
• react
• remote
• simulation-mode
• stt
• tts
• twilio
• typescript
• vercel
• vercel-analytics
• voiceai
• webaudioapi
• webmcp
• websockets
• youtube