FlakyLens

FlakyLens is an AI-powered QA automation tool that tackles one of the biggest pain points in modern test suites—flaky tests. Designed with real-world DevOps workflows in mind, FlakyLens intelligently detects, explains, and even suggests fixes for flaky tests, saving QA engineers hours of debugging and guesswork.

Built with Node.js, TypeScript, and integrated into CI/CD via GitHub Actions or GitLab, FlakyLens parses test logs, identifies statistically flaky tests, and leverages OpenAI’s language models to provide human-readable diagnoses and repair suggestions. It includes a real-time dashboard for test health monitoring and offers CLI-based interactions for seamless developer workflows.

FlakyLens goes beyond static analysis or retries—it empowers teams with actionable insights, integrates directly with your pipeline, and scales effortlessly. It’s like having an AI test architect watching over your test suite 24/7.

Whether you're running tests in Playwright, Cypress, or Selenium, FlakyLens is your smarter path to a stable, trustworthy testing future.

Problem: Flaky tests are a widespread problem in automated test suites. They: Fail inconsistently without code changes Erode trust in test results Waste debugging time Slow down CI/CD pipelines

Most QA teams address flakiness manually, relying on guesswork or silencing tests. There's a need for an intelligent tool that detects, explains, and even auto-heals flakiness.

✅ Our Solution: FlakyLens A lightweight, plug-and-play CLI + dashboard tool that automatically:

• Detects flaky tests using historical CI logs + test retry data.
• Diagnoses causes using AI models trained on flaky test patterns.
• Suggests fixes (e.g., add waits, isolate test data, mock network).
• Auto-annotates flaky tests in code with actionable warnings.

Generates a report dashboard for your team and PR checks.

🔧 Tech Stack

• Language: TypeScript + Node.js
• Testing frameworks: Playwright, Cypress (extensible to others)
• AI/ML: OpenAI GPT-4 for pattern recognition & explanations
• Backend: Express (for dashboard), SQLite (for local log DB)
• CI Integration: GitHub Actions / GitLab CI via CLI tool
• Visualization: React + Chart.js dashboard

🛠️ Key Features (Project Code) flakylens analyze – CLI command to scan test history (local or CI logs), detect flakiness (e.g., 2/5 pass rate), and categorize cause (timing, network, async bugs). flakylens explain <testName> – AI-assisted explanation of why a test is flaky. flakylens suggest-fix – Proposes code adjustments based on test symptoms. flakylens dashboard– Launches a local dashboard showing: Flaky test frequency Root causes by category Suggested fixes and trends over time Integration Hooks – Easily added to CI pipelines to fail builds if flaky tests exceed threshold or auto-label PRs with needs-attention.

Project Code Structure:

flakylens/
├── cli/
│   └── index.ts             # Main CLI interface
├── core/
│   ├── flakinessDetector.ts # Detects flaky tests
│   ├── logParser.ts         # Parses test output logs
│   └── aiAdvisor.ts         # Uses OpenAI API for suggestions
├── dashboard/
│   ├── server.ts            # Express server
│   └── frontend/            # React dashboard
├── db/
│   └── history.sqlite       # Local DB for test history
├── tests/
│   └── sampleTestLogs/      # Example logs for demo
├── .github/
│   └── workflows/ci.yml     # CI integration example
├── README.md
└── package.json

Built With

• chart.js
• express.js
• gpt-4
• node.js
• sqlite
• typescript