TestDino

AI-native Playwright test reporting with MCP support. Classify failures, track flaky tests, and debug runs using Claude or Cursor in natural language.

Comment

Inspiration Most Playwright teams we worked with were spending hours every day manually triaging CI failures, hunting through artifacts, and guessing whether a test was genuinely broken or just flaky. Existing tools either gave you a static HTML report or charged enterprise prices for basic visibility. We wanted to build something that actually understood Playwright workflows from the ground up and brought intelligence to the debugging process rather than just displaying raw results.

What it does TestDino is an AI-native Playwright test reporting and management platform. It classifies failures with confidence scores and fix suggestions, tracks flaky tests systematically across branches and environments, posts AI summaries directly to GitHub PRs, blocks bad merges through CI checks, and lets developers query their entire test history using natural language through Claude Code, Cursor, or any MCP-compatible LLM tool.

How we built it We built it on Next.js, Node.js, and TypeScript with PostgreSQL for structured test data and S3 for artifact storage. The AI layer uses OpenAI and Anthropic APIs for failure classification and natural language querying. GitHub, Jira, Linear, Asana, and Slack integrations are built on their respective APIs. MCP support enables LLM tools to interact directly with test run data.

Challenges we ran into Getting AI failure classification accurate enough to be trusted by engineers was the hardest problem. False confidence scores would erode trust faster than no classification at all. Handling large trace files efficiently without slowing down the reporting pipeline was another significant infrastructure challenge.

Accomplishments that we are proud of Teams using TestDino have reduced flaky tests from 30+ down to 3 or 4 and cut CI reporting costs by 30 to 40%. Saving engineers hours of daily manual triage is the outcome we set out to achieve and seeing it happen in production is the most meaningful validation we have received so far.

What we learned Engineers do not just want more data. They want fewer decisions to make. The most impactful features were not the ones that showed the most information but the ones that removed a manual step entirely. Simplicity in how insights are surfaced matters more than the depth of the underlying analysis.

What's next for TestDino Deeper MCP integrations, expanded support for multi-framework test suites beyond Playwright, smarter rerun logic that predicts which tests are likely to fail before a full CI run completes, and more integrations with cloud testing infrastructure providers.

Built With

Share this project:

Updates