OpenTester

• 

  logo

Inspiration

I've been an Android engineer for 9 years. Coding agents like Cursor and Claude Code have completely transformed how fast I can write code.

But there's a new bottleneck: testing.

Every time the agent writes code, I still have to:

• Navigate the same screens
• Verify the feature works
• Re-run regression tests to ensure nothing else broke
• Repeat and repeat

So even though agents can write code autonomously, a human still has to verify everything manually.

That's why I built OpenTester, an AI that connects to coding agents via MCP and automatically generates and runs end-to-end tests, just as a human would.

What it does

OpenTester is an AI QA engineer for coding agents like Cursor or Claude Code.

When a coding agent implements a feature, OpenTester automatically launches the Android app, navigates it like a human, and verifies that the expected behavior works correctly.

While exploring the app, OpenTester records every interaction required to complete the flow.

These interactions are saved as human-readable JSON tests, creating a regression suite that can be replayed later to ensure new changes don't break existing features.

How we built it

We built a local MCP server in TypeScript that exposes (3) tools to coding agents:

• generate_tests: navigates the app and records a test in JSON format
• validate_test: replays a test and returns pass or fail
• list_tests: lists available test files

The MCP server connects to an Android device via ADB, allowing it to: capture screenshots, read the UI accessibility tree, perform actions like tap, scroll, type, and back

To plan and execute tests, the MCP server communicates with a multi-agent backend on GCP Cloud Run built with Google ADK.

Pipeline

1. The coding agent sends a test goal (e.g. “log in and verify the home screen”).
2. A Planner agent breaks the goal into smaller UI steps.
3. The system repeatedly:
   • captures the screen and UI tree
   • lets an AI agent decide the next action
   • executes the action on the device
4. Each step is recorded

When the flow finishes, OpenTester generates a JSON test file describing the full UI interaction. These tests can later be replayed deterministically without AI for regression testing.

Challenges we ran into

1. Mobile navigation is inherently unpredictable. Unexpected permission prompts and system pop-ups often break traditional automation. We addressed this by capturing these edge cases and incorporating them directly into the agent’s contextual prompts. This allows the agent to navigate fluidly and handle "interruption" states with human-like reasoning.

2. Limitations of pure vision. Our initial approach relied solely on screenshot analysis, which proved fragile. The agent struggled with precise coordinate mapping, and tests frequently broke when scaled across different screen resolutions.

The Solution: We use screenshots together with the view hierarchy, so the agent now maps visual intent to structured UI elements. This multi-modal approach ensures the agent reliably identifies targets (buttons, scrollers, etc.) regardless of screen size or aspect ratio.

Accomplishments that we're proud of

1. Easy set-up: we published an official npm package, enabling developers to integrate our MCP server into their coding agents in a simple way.

2. Resilient UI Navigation: Our agent demonstrated high reliability across diverse applications, successfully navigating complex edge cases and recovering from intrusive elements like unexpected permission pop-ups.

3. Standardized Testing Framework: we established a human-readable JSON-based schema for defining Android UI flows, allowing AI agents to reproduce previous tests in an easy way without code.

What we learned

1. ADK is powerful. ADK saved us a lot of development time thanks to its simple abstractions. Features like InMemorySessionService for session management and before_model_callback for multimodal input injection made it easy to build complex agent workflows quickly.

2. The technology is almost there. Using Gemini models, we achieved strong results on complex UI navigation tasks. This suggests that AI is close to being able to autonomously test Android apps without human supervision.

What's next for OpenTester

1. Business model. OpenTester will follow a developer-first model: it will remain free and open-source, and developers can use it in a BYOK (Bring Your Own Key) way. Devs and teams will pay for cloud execution, CI integrations, parallel device testing, and large-scale regression runs.

2. CI/CD integrations. Integrate with GitHub, CI pipelines, and pull request workflows so tests run automatically before code is merged.

3. Cross-platform support. Expand beyond Android to support iOS and web applications, enabling end-to-end testing across entire products.

Built With

• adk
• python
• typescript