Kodez CheckMate

Kodez CheckMate securely automates Auth0 SecOps: run Auth0 CheckMate security scans, review findings with AI and, with approval, create JIRA/Confluence updates and Teams alerts via Auth0 TokenVault.

Comment

Inspiration

We are heavy users of Auth0 for identity management, and keeping our Auth0 tenant's security posture healthy is a constant priority. When Auth0 introduced the Auth0 CheckMate tool — a scanner that quickly identifies misconfigurations and security hotspots across your tenant — it was immediately useful, but it only gave us a snapshot. There was no built-in way to track findings over time, assign remediation ownership, communicate results to stakeholders, or follow issues through to closure.

We needed an orchestration layer on top of CheckMate, one that could close the loop between a scan result and an actioned fix. We also wanted to bring AI into the picture to reduce the manual effort our developers spend managing security workflows.

Around the same time, we discovered the AuthorizedToAct Hackathon — and the timing felt like a signal. We had a real problem, a clear solution in mind, and a platform to build it on. That convergence is how Kodez CheckMate was born. What started as a hackathon submission has since become a platform we use internally every day.

What It Does

Kodez CheckMate is an AI-powered SecOps orchestration platform that takes Auth0 CheckMate security scans from raw results to tracked, documented, and actioned findings — driven entirely by natural language prompts.

At its core, the platform connects your Auth0 tenant to the tools your team already uses: JIRA, Confluence, Microsoft Teams, and GitHub. A GPT-powered AI Agent sits at the centre of the experience, letting you trigger scans, query results, and coordinate remediation workflows through a conversational interface rather than a series of manual steps across disconnected tools.

Key capabilities include:

  • Prompt-driven scanning — trigger Auth0 CheckMate scans with a single natural language instruction
  • Human-in-the-loop approvals — every agent action requires explicit user sign-off before it executes
  • Multi-task chaining — combine scan, document, ticket, and alert into one prompt and let the agent handle the sequencing
  • Scan visualisation and comparison — charts showing vulnerability trends across scans, with on-demand delta comparisons
  • Automated JIRA ticket creation — structured remediation tickets generated for every open finding, with no copy-pasting
  • Confluence executive summaries — publish structured scan reports directly to a Confluence page for audit trails and stakeholder visibility
  • Microsoft Teams alerts — push real-time vulnerability notifications to the right channel the moment new issues are found
  • Enhanced PDF reports — export polished, structured scan reports with severity breakdowns and remediation recommendations
  • Secure credential management via Auth0 Token Vault — all integration tokens stored exclusively inside Auth0's infrastructure, never exposed to the client application

How We Built It

Kodez CheckMate is built as a full-stack platform with the following components working in concert:

Authentication is handled entirely through Auth0. Users log in with their Auth0 account, and the integration connections — GitHub, JIRA and Confluence — are established through Auth0 social connections. Microsoft Teams connection is established via Microsoft Azure AD Enterprise Connection. GitHub had a native out-of-the-box connector. For JIRA and Confluence, we registered them as custom OIDC-compliant social connections, since Auth0 supports any OIDC-compatible identity provider. This kept the entire authentication and token lifecycle within the same Auth0 trust boundary.

Token security is managed through Auth0 Token Vault. Every OAuth access token and refresh token issued by the connected integrations is stored inside the vault, completely isolated from the client application. The platform never holds a token directly — when the AI Agent needs to call an external service, it routes through the vault. This eliminated an entire class of credential exposure risk from the architecture.

Scan execution is triggered via a GitHub Workflow. When the AI Agent receives a scan instruction, the platform dispatches a trigger to a GitHub Actions workflow, which executes the Auth0 CheckMate scanner against the target Auth0 tenant. The scanner pulls its configuration directly from the live Auth0 Tenant, ensuring every scan reflects the current state of the identity environment. Results are returned to the platform and persisted to the database.

The AI Agent is powered by GPT-5.4 running on Azure Foundry. The agent is the primary interface — not a supplementary feature. It parses natural language prompts, identifies the intended actions, presents a confirmation step to the user, and then executes each task in sequence. Multi-step prompt chaining allows complex end-to-end workflows to be initiated in a single conversation turn.

Downstream integrations — JIRA ticket creation, Confluence page publishing, and Microsoft Teams alerts — are all invoked by the AI Agent using credentials securely retrieved through Auth0 Token Vault at the point of use.

Challenges We Ran Into

The most significant challenge we faced was secure management of third-party access tokens. Kodez CheckMate integrates with four external services — GitHub, JIRA, Confluence, and Microsoft Teams — each of which issues OAuth access tokens and refresh tokens that carry real write permissions. Handling these credentials safely in a multi-integration environment is non-trivial.

The naive approaches all had unacceptable trade-offs. Storing tokens in a database introduced a high-value target for credential theft. Passing them through the application layer created exposure in transit. Caching them in the browser was a clear non-starter. A dedicated secrets manager was operationally complex and would have placed a burden on customers to manage.

Auth0 Token Vault resolved this cleanly. By storing every credential inside Auth0's infrastructure and routing all token usage through the vault, we eliminated the client application from the credential chain entirely. The app never holds a token. The browser never sees one. This was the architectural decision that took the most deliberation — and ultimately the one we would change least.

A secondary challenge was the OIDC integration for services without native Auth0 social connections. GitHub and Microsoft Teams were straightforward, but JIRA and Confluence required registering custom OIDC-compliant connections. Working through the nuances of each service's OIDC implementation and ensuring token lifecycle management flowed correctly into Token Vault took careful testing across all three.

Accomplishments That We're Proud Of

The accomplishment we are most proud of is the successful integration of Auth0 Token Vault into a real-world, multi-integration production architecture. Designing a system where four external services — each with its own OAuth flow — are connected, authenticated, and operated entirely through Auth0's trust boundary, with no credentials ever touching the client application, is a meaningful security achievement. It is not a theoretical model; it is a working platform we use every day.

Equally, we are proud of delivering a fully functional AI-powered application that goes beyond a demo. The AI Agent genuinely orchestrates end-to-end SecOps workflows — from triggering a scan to filing JIRA tickets to publishing a Confluence summary and alerting the team on Teams — in a single conversation, with human-in-the-loop approval at every step. That combination of real utility, production-grade security, and AI-driven automation in a single cohesive platform is something we are genuinely proud to have shipped.

What We Learned

The most substantive thing we learned through building Kodez CheckMate is how to securely integrate Auth0 Token Vault into a real-world application with multiple third-party OAuth connections.

Token Vault is more than a convenience — it is a fundamentally different mental model for thinking about credential ownership in a platform. Understanding how to structure OIDC connections so that token issuance and refresh flows directly through Auth0, how to route agent-initiated API calls through the vault without exposing credentials at any layer of the stack, and how to reason about the compliance implications of that architecture for frameworks like SOC 2 and ISO 27001 — all of that was hard-won knowledge that will shape how we build secure integrations going forward.

We also developed a much deeper appreciation for Auth0's OIDC extensibility. The fact that any OIDC-compliant provider can be onboarded as a social connection — and immediately benefit from Auth0's token lifecycle management and Token Vault integration — is a genuinely powerful capability that is easy to underestimate until you build against it.

What's Next for Kodez CheckMate

The immediate priority is evolving Kodez CheckMate into a fully-fledged SaaS offering available to external customers. We want to make it straightforward for any organisation running Auth0 to connect their tenant, run their first scan, and start managing their identity security posture without any complex onboarding. The goal is to become the operational layer that sits on top of Auth0 CheckMate for teams who need more than a point-in-time report.

On the AI side, we plan to introduce more guardrails around the LLM integration to prevent misuse and ensure the agent operates strictly within the boundaries of its intended SecOps role. This includes tighter prompt validation, action scope controls, and audit logging for every agent-initiated operation.

Longer term, we see Kodez CheckMate expanding beyond Auth0 CheckMate scans to address a broader set of identity security and compliance monitoring use cases — helping teams not only find misconfigurations, but continuously maintain a healthy IAM posture across their entire identity infrastructure.

Bonus Blog Post

The content of this blog post is also available at: https://authorizedtoact.kodez.au/blog.html

Introducing Kodez CheckMate: AI-Powered SecOps Orchestration for Auth0

Security operations teams face a familiar problem. Vulnerability scanners surface findings. Those findings get copied into spreadsheets, pasted into tickets, summarised in emails, and posted in Slack channels — by hand, one step at a time. The tools exist, but the glue between them does not. Teams end up spending more time on the workflow around a vulnerability than on actually fixing it.

Kodez CheckMate is built to close that gap. It is an AI-powered SecOps orchestration platform that takes Auth0 CheckMate security scans from raw results to tracked, documented, and actioned findings, driven entirely by natural language prompts.

What Is Auth0 CheckMate?

Auth0 CheckMate is a security scanning tool built specifically for Auth0 tenants. It inspects your Auth0 configuration and surfaces vulnerabilities, misconfigurations, and compliance risks across your identity infrastructure.

The out-of-the-box CheckMate tool gives you a point-in-time report. That is useful, but it is just the starting point. The real work — triaging findings, creating tickets, communicating to stakeholders, and tracking remediation over time — happens elsewhere, and until now that handoff has been entirely manual.

Kodez CheckMate changes that.

Kodez CheckMate Architecture

At the centre is the Kodez CheckMate platform, which coordinates the AI Agent, Database, and Auth0 Token Vault. The AI Agent delegates LLM inference to Azure Foundry GPT and drives the Auth0 CheckMate security scanner. The scanner receives its scan configuration directly from the Auth0 Tenant — the live identity environment being assessed — ensuring every scan reflects the actual state of your Auth0 configuration. Results are persisted to the database and surfaced through the downstream connectors: JIRA, Confluence, and Microsoft Teams. GitHub Workflow sits apart from these — its sole role is to trigger the Auth0 CheckMate security scanner, kicking off scans on demand or on a schedule.

How It Works

Log In with Auth0

Everything starts with your Auth0 account. Users authenticate directly through Auth0, so there is no separate credential to manage and no new identity silo to introduce into your stack.

Once logged in, you connect the integrations your team already uses: GitHub, JIRA, Confluence, and Microsoft Teams. The connection process is built on top of Auth0 Token Vault, which means the access tokens and refresh tokens issued by each integration are stored securely inside the vault, completely out of reach of the client application. Tokens are never exposed in transit, never held in your browser, and never accessible outside the vault environment. This is not just a security convenience — it is a meaningful reduction in token misuse risk and a genuine compliance improvement for teams operating under frameworks like SOC 2, ISO 27001, or similar.

Why We Built Auth0 Token Vault Into Kodez CheckMate From Day One

When we started building Kodez CheckMate, the hardest problem was not the AI Agent, the scan orchestration, or the JIRA integration. It was a question we kept coming back to: where do the tokens live?

CheckMate connects to GitHub, JIRA, Confluence, and Microsoft Teams on your behalf. That means OAuth access tokens, refresh tokens, and credentials with real write permissions. The naive path — storing them in a database, passing them through the app layer, caching them in the browser — felt wrong from the first whiteboard session. Not wrong in a theoretical way. Wrong in a this will cause an incident one day way.

We evaluated several approaches. Environment variables felt too brittle at scale. A secrets manager added operational overhead we did not want to impose on users. Then we looked harder at Auth0 Token Vault, and something clicked.

Connecting the Integrations: Out-of-the-Box and OIDC

Before we even got to Token Vault, we had to solve a related challenge: Auth0 does not have a social connection for every service we needed. GitHub was straightforward — Auth0 ships with a GitHub connector and Microsoft Teams connector (Azure AD Enterprise) out of the box. JIRA and Confluence were a different story.

What made this tractable is that Auth0 supports any OIDC-compliant connection. Each of those three services exposes an OIDC-compatible identity layer, so we registered them as custom social connections and Auth0 handled the rest — including the token lifecycle. No custom OAuth dance, no hand-rolled callback handlers. If Auth0 does not have a built-in connector for a service you need, the answer is straightforward: if it speaks OIDC, you can connect it.

Key insight: Auth0's OIDC support means you are not limited to the services with first-party social connections. Any OIDC-compliant provider can be brought into the same authentication and token management model — which is exactly how we integrated JIRA and Confluence.

Token Vault: Closing the Exposure Gap

That is where Token Vault came in. It keeps every credential inside the Auth0 infrastructure, completely isolated from the client application. The app never holds a token. The browser never sees one. When the AI Agent needs to create a JIRA ticket or post to Teams, it calls through the vault — the token is used, but never exposed. The attack surface for credential theft essentially collapses.

What surprised us was how naturally this composed with Auth0's existing session model. Users were already authenticating through Auth0 to access CheckMate. Managing integration credentials through the same trust boundary — OIDC connections feeding directly into Token Vault — felt like a coherent system, not a patchwork of loosely connected parts.

The compliance angle mattered too. Teams operating under SOC 2 or ISO 27001 need demonstrable controls around third-party credentials. Token Vault provided that story without adding any operational burden to customers. The architectural decision that took the most deliberation ended up being the one we would change least.

The AI Agent: GPT-5.4 on Azure

At the heart of the platform is an AI Agent powered by GPT-5.4, running on Azure Cloud. This is not a chatbot bolted onto a dashboard — it is the primary interface for the platform. You interact with it through natural language prompts, and it handles the orchestration behind the scenes.

The agent understands the context of the platform. When you type Run security scan, it recognises the intent, identifies the appropriate action — invoking Auth0 CheckMate against your target Auth0 tenant — and responds with a clear description of what it is about to do, asking you to confirm before proceeding. This human-in-the-loop approval step is a deliberate design choice: every action the agent takes requires explicit user sign-off, so nothing runs without your knowledge.

Prompt Templates for Faster Interaction

To speed up common workflows, the platform provides a set of pre-defined prompt templates directly in the agent chat. Rather than typing from scratch, users can pick a template — Run security scan, Compare with last scan, Create JIRA tickets for all findings, Publish Confluence summary, Alert team on Teams — and the agent is ready to act immediately.

Chain Multiple Tasks in a Single Prompt

One of the more powerful capabilities is task chaining. Instead of triggering actions one by one, you can combine multiple tasks into a single prompt:

Run security scan. Publish the executive summary into Confluence. Create JIRA tickets for issue tracking and update the stakeholders via Teams.

The agent parses that as four distinct actions, presents them to you for approval in sequence, and then executes each one in order — without you having to re-engage between steps.

Security Scans, Visualised

After a scan completes, the agent does not just return a wall of JSON. It processes the results and presents them visually — charts and graphs that show the number of open vulnerabilities per scan, so you can immediately see whether your security posture is improving, degrading, or holding steady over time.

You can also compare scans directly. Ask the agent How do today's results compare to last week's scan? and it will surface the delta: new findings, resolved issues, and anything that has changed.

From Findings to JIRA Tickets, Automatically

Once you have scan results, the next step is usually getting those findings into your project management workflow. With Kodez CheckMate, that is a single prompt:

Create a JIRA board and create tickets for all findings.

The agent connects to your linked JIRA account via the integration and creates structured tickets for every open vulnerability — no copy-pasting, no manual triage, and no context switching.

Executive Summaries in Confluence

Security findings do not just need to be actioned — they need to be documented. With one prompt, the agent can publish an executive summary of your scan results directly to a Confluence page. The content is structured and readable, giving leadership and audit teams the visibility they need without requiring engineers to write reports manually.

Real-Time Alerts in Microsoft Teams

Critical vulnerabilities should not wait for the next standup. When new issues are identified in your Auth0 tenant, you can prompt the agent to push alerts directly to a Microsoft Teams channel so your security and engineering teams get notified immediately with actionable context.

Enhanced PDF Reports

For teams that need formal documentation, Kodez CheckMate includes a PDF export capability that goes beyond the default Auth0 CheckMate report. The exported reports are structured for clarity with organised findings, severity breakdowns, remediation recommendations, and scan comparisons in a clean, professional format.

Why It Matters

The individual pieces here — security scanning, ticketing, documentation, and alerting — are not new. What is new is the orchestration layer that connects them, driven by a model that understands what you are trying to do and handles the coordination automatically.

For security teams, that means less time managing workflows and more time reducing risk. For engineering teams, it means findings arrive in their existing tools in the right format without manual handoff. For leadership, it means visibility into security posture without waiting for a weekly report.

Kodez CheckMate is available now. Log in with your Auth0 account to connect your integrations and run your first scan.

Kodez CheckMate — AI-Powered SecOps Orchestration for Auth0.

Built With

Share this project:

Updates