Rehearsal

An open-source MCP testing workbench for evaluating tool schemas, voice-first usability, and deterministic tool-routing performance through explainable heuristics and reproducible tests.

Comment

Inspiration

Voice assistants depend on well-designed tools to interpret user intent and return useful responses. However, MCP (Model Context Protocol) tools can expose deeply nested schemas, excessive parameters, technical identifiers, and outputs that are difficult to understand when spoken aloud.

We built Rehearsal to explore how tool design affects voice-first usability. Our goal was to create a practical testing workbench that helps developers identify schema and response-design issues before integrating tools into voice-driven applications.

What It Does

Rehearsal is an independent, open-source MCP Voice Testing Workbench that compares flawed tool designs with voice-first alternatives.

The platform provides:

  • Tool Explorer: Inspect six MCP reference tools, their schemas, and their design differences.
  • Voice Linter: Apply five heuristic rules to identify schema complexity, excessive arguments, TTS-hostile output, parameter ergonomics, and unclear descriptions.
  • Benchmark Comparison: Compare flawed and corrected tool designs across Weather, Smart Home, and Travel domains.
  • Evaluation Runner: Execute 18 deterministic test fixtures for tool selection and argument extraction.
  • Run History: Persist and inspect evaluation results through a SQLite-backed API.
  • MCP Integration: Use the official Python MCP SDK in the reference-server and client workflow.

How We Built It

We built the frontend with React, TypeScript, and Vite, and the backend with Python, FastAPI, and SQLite. The project includes a reference MCP server, a rule-based voice-readiness linter, a deterministic evaluation runner, API endpoints, and an interactive developer workbench.

We designed paired flawed and voice-first tool examples for Weather, Smart Home, and Travel. The linter applies five explainable heuristic rules, while the evaluation runner executes predefined fixtures and records the results for later inspection.

We also developed automated backend tests and documented the architecture, setup instructions, limitations, and live demonstration workflow.

Challenges We Faced

One major challenge was turning subjective voice-usability concerns into understandable, repeatable checks. We addressed this by defining explicit heuristic rules and pairing flawed tool designs with corrected alternatives.

Another challenge was keeping benchmark comparisons consistent and ensuring that evaluation results could be reproduced and inspected. We used deterministic fixtures, API validation, automated tests, and persistent run history.

We also had to distinguish simulated evaluation from real-world voice performance. The current fixture suite does not measure acoustic noise, microphone quality, regional accents, or open-ended conversations.

What We Learned

We learned how MCP tool schemas, parameter design, and spoken-response formatting can be evaluated through explicit heuristics. We also gained practical experience building a full-stack developer tool, integrating MCP workflows, designing deterministic tests, and maintaining an auditable evaluation history.

Our results are a starting point for systematic testing, not proof of real-world voice-assistant accuracy.

What's Next

Future work could include integration with real speech-recognition pipelines, broader test datasets, controlled user studies, and evaluation across different speech-recognition and language-model systems. These extensions would help determine how well the current heuristics generalize beyond deterministic fixtures.

Project Scope

Rehearsal is an independent open-source project. It is not affiliated with, endorsed by, or an official implementation of Amazon Alexa or Alexa+.

Built With

Share this project:

Updates

Submission history