Pipeline variables necessary for PHI Trace to work in Bitbucket
Example Vendor Register (No BAA w/ Acme) in Confluence
Example Failing Build (No BAA w/ Acme) in Bitbucket
Example Vendor Register (Has BAA w/ Acme) in Confluence
Example Passing Build (Has BAA w/ Acme) in Bitbucket
Example Custom Policy (Logging lastLabResult) in Confluence
Example Failing Build (Logging lastLabResult) in Bitbucket
Example Output after a Build in Confluence
Merge blocked on premium plans

What it does

PHI Trace flags potential patient data exposures in Bitbucket pull requests and checks them against rules your team keeps in Confluence.

Vendor checks: if a pull request sends patient data to a vendor, PHI Trace checks your Vendor Register and cites what it finds, such as no BAA on file.
Team rules: compliance adds a row to a table on a labeled Confluence page, and the next pull request that breaks it fails, with no code change.
Results where people work: findings appear inline on the pull request, high-risk findings fail the build, and an audit report in Confluence updates itself. On Bitbucket Premium, a failed build can also block the merge.
A Rovo agent: it explains each finding and suggests a fix, and it turns a plain-English rule into a draft row for a person to review.

What inspired me

General security tools look for unsafe code, but a lot of patient data exposure comes from code that works exactly as written such as a log line that includes a lab result, or an analytics call that includes a patient's name. Whether that's acceptable often depends on things outside the code, like which vendors have a BAA and what the compliance team has decided. Those answers usually live in a place like Confluence, far from the code itself and I wanted to connect them, especially for smaller teams with no dedicated security personnel making sure all of the bases are covered.

How I built it

PHI Trace has two parts, connected by a Forge web trigger.

The scanner runs in Bitbucket Pipelines. It downloads its rules from the app, adds the team's rules from Confluence, and runs Semgrep to follow patient data from where it's read to where it leaks. Only the rule, severity, file, line and vendor are sent to the app, never the source code.

The Forge app runs on Confluence. It reads the Vendor Register and labeled policy pages, stores findings, writes the audit report, powers the Rovo agent, and provides a setup page. Each install generates its own secret, and every request is checked against it to ensure proper authorization.

Detection is built from a reference file with a pack for each vendor, which teams can extend with a phi-trace.yml file. Team rules from Confluence are validated and turned into Semgrep rules, so a typed table row can't inject anything into a rule.

Challenges

The architecture changed partway through. I started with a Bitbucket merge check, then learned that one Forge app can't combine Bitbucket and Confluence permissions, and that Forge Rovo agents don't run in Bitbucket. Asking teams for two installs felt like too much, so I dropped the Bitbucket app and made the pipeline the connection between the products. It came out simpler, and a failed build still blocks merges on Premium with Bitbucket's own settings.

Deciding how to use AI. I tested a Forge LLM on labeled cases for finding flaws in code. It caught a lot, but its false alarms missed the bar I had set for myself when building, and I wanted results I could repeat and verify, with no code sent to a model. Semgrep enforces every rule. AI's role is explaining findings and drafting rules from a person's own words via Rovo.

Not fooling myself with test scores. My first rules scored 100% on the cases I wrote them against, then caught only 3 of 17 flaws on new cases. Writing new test sets before changing the rules kept me honest. It also changed the product: a list of names can't cover every vendor, so PHI Trace became a dictionary teams can extend, and I'm upfront about what it can't see or do.

What I learned

Checking platform limits before designing around them, since one early test could have saved me a rebuild!
A tool that reads code and a person who knows the rules are stronger together. Letting compliance write enforceable rules without an engineer or security team became the heart of the product.
For a tool like this, being clear about its limits matters. PHI Trace flags potential exposures for a team to assess, and doesn't determine compliance or whether a breach has actually occurred.