Tester App

Developers dont have time to unit test all the code comprehensively and its also expensive to maintain a lot of tests in codebase. The solution: short lived dynamic unit tests which runs on merge

Comment

Tester-App — Dynamic Unit Tests on Merge

Modern teams can’t afford to write and maintain exhaustive unit tests for every edge case. Tester-App generates short-lived, PR-scoped unit tests from the diff and deletes them after the run. The goal: catch regressions early without growing a brittle, ever-expanding test suite.

Why now?

Velocity vs. safety: fast releases but rising bug risk.

Test rot: maintaining thousands of tests is expensive.

Diff-aware guardrails: 80% of PR bugs surface in changed files and their call-sites.

What it does

Analyze the PR diff to identify changed functions, public methods, and risky code paths.

Synthesize focused unit tests (optionally LLM-assisted) into a temp workspace:

Golden-path assertions for new/edited functions.

Edge-case inputs inferred from type hints, guards, and branches.

Contract tests for public methods touched by the diff.

Run tests in CI (pytest/Jest/etc.), annotate failures back to the PR, upload artifacts.

Auto-dispose: ephemeral tests are not committed to the repo.

What it is not

A replacement for your core regression suite or domain-critical tests.

A magic bullet—the generator is deterministic + heuristic, with optional LLM for richer cases.

How it works (pipeline)

on: pull_request

Diff Analyzer Parses the PR, builds a dependency graph (imports/call graph), picks the blast radius.

Spec Extractor Gleans contracts from type hints, docstrings, guard clauses, and examples in code/comments.

Test Synthesizer Emits ephemeral tests:

Value sweeps around boundaries (0/±1, empty/None, max/min).

Error/guard cases (exceptions, early returns).

Property checks for pure functions (invariants, idempotence).

(Optional) LLM prompt to produce richer cases + table-driven tests.

Runner

Spins a sandbox (venv) and executes pytest (or framework-specific runner), collecting junit + coverage.

Reporter

Posts PR annotations, summary comment, and uploads generated tests as artifacts.

Cleanup

Deletes temp workspace. No changes committed.

Challenges ran into :

No egress to go back and forth with open AI

Built With

Share this project:

Updates