-
-
TestLens: See what your tests actually prove.
-
Review the assertion, execution evidence and scope before accepting a criterion’s coverage.
-
Four of six criteria are now verified within the reviewed scope. External checks and an undefined requirement remain open.
-
Removing the capacity guard leaves the original tests passing. Two additional behavioural checks catch the defect.
-
Three tests pass, but the required booking behaviour remains unverified. TestLens shows what the test checks and what is missing.
Inspiration
A green build is reassuring. It still leaves a fairly important question: what did the tests actually check?
A test can confirm that a capacity function returns false without checking that the application rejects a booking or avoids sending a request. The test passes. The requirement is only partly covered.
I built TestLens to make that gap visible inside Jira, where the requirement and the decision about what to test next already live.
What it does
TestLens connects acceptance criteria to reviewed test assertions, source snapshots and execution evidence. For each criterion, you can inspect the expected behaviour, what the test checks, the matching result and what remains uncertain.
A passing report alone does not make a criterion verified. Mapping a test to a requirement needs human review. Missing assertions, failed attempts, stale requirements and external checks remain visible.
The next-test recommendation explains which behaviour needs attention and why. You can turn it into a linked Jira task after reviewing the proposed content. Completing that task does not automatically verify the requirement.
Rovo can explain the same evidence and suggest what to inspect next. Its TestLens actions are read-only and check the requesting user’s Jira access. Project briefings include accessible issues without evidence, so unassessed work does not disappear from the picture.
How I built it
TestLens runs on Atlassian Forge, with a React and TypeScript Custom UI, Jira issue and project modules, and installation-scoped Forge storage. A shared evidence engine computes the decisions used by the interface and Rovo.
The optional GitHub Actions connection accepts evidence only from an explicitly approved repository, workflow, branch and exact commit. It verifies short-lived GitHub identity tokens and records the accepted run. This identifies the approved producer; it cannot prove that a dishonest producer reported the truth.
Uploaded code is never executed inside Forge. Evidence stays in Forge storage. The optional GitHub integration fetches public signing keys rather than sending the evidence to an external database.
Challenges I ran into
The difficult part was keeping the meaning of “verified” narrow enough to be useful. A source hash is not an execution record. A test name is not an assertion. A successful retry does not erase the failed attempt.
The application also needs to behave sensibly when a requirement changes, a connection is revoked or two people review the same evidence. Those cases can remove current verification without rewriting the recorded test result.
Live Jira testing mattered. It exposed an accepted mapping incorrectly labelled as changed after storage reordered its fields, and keyboard focus escaping an open dialog into Jira. Both now have regression tests and verified fixes.
Accomplishments I’m proud of
The clearest demonstration is deliberately small. In the synthetic booking app, removing the capacity guard leaves all three original tests passing. Two additional behaviour checks catch the defect. Restoring the guard makes all five pass.
After reviewing the new mappings, TestLens moves from two to four verified criteria out of six. External email delivery and an undefined cutoff still remain open. That is the distinction I wanted the product to preserve.
The current release passed 162 unit and integration tests and 109 browser checks, with two documented skips. It has also been installed through the judge link on a fresh Jira site and tested with a separate restricted account. These checks support the release; they are not a claim that the product is defect-free.
What I learned
The explanation beside a status matters as much as the status itself. “Not verified” needs to tell someone what is missing and what to investigate next. Otherwise it is just another coloured badge.
I also learned to test the embedded experience early. A dialog that appears to work in a standalone preview can behave differently inside Jira.
What’s next for TestLens
The next step is a small pilot with developers and QA engineers working on their own requirements. I want to find out whether the evidence trail changes which tests they write and whether the review effort is worthwhile.
Larger evidence storage, retention controls and additional CI providers are possible follow-ups. Marketplace distribution and billing are future work. For now, the priority is making one decision clearer: which behaviour needs attention next, and what evidence supports that choice?
Built With
- atlassian-forge
- forge-kvs
- github-actions
- jira
- junit-xml
- playwright
- react
- rovo
- typescript
- vitest
