GitHub Secrets Detection in XSOAR

An XSOAR integration to search through the GitHub repositories for secrets in commit history.

Comment

Inspiration

Shift-left security is a practice to embrace security to the software at the earliest stage of the development process by performing static code analysis, vulnerability scans and also secret detection to avoid data leak. Most of these are set up by the DevOps and developers with different levels, tools, priorities and schedules in the developer Git pre-commit hooks or within the CICD pipeline.

Can we have an XSOAR automation pack to quickly configure a GitHub repository URL, do the scans dynamically with new commits in a branch, and create incidents for the SOC department to investigate?

What it does

I have created a XSOAR automation pack for GitHub Secrets Detection. This is the list of the features:

  1. It can automatically detect the new commits in a branch of a repository.
  2. It supports GitHub.com and GitHub Enterprise.
  3. It can scan all the commit histories of an existing repository.
  4. It supports customized file name and secret key patterns.

How I built it

First, I was using demisto-sdk to scaffold an empty project. The integration is written in Python. To detect the secrets in the file content and certain file names, I am integrating truffleHog (https://github.com/dxa4481/truffleHog) and shhgit (https://github.com/eth0izzle/shhgit/blob/master/config.yaml) that has a nice list of patterns for the file name and path.

Challenges I ran into

  1. There are many secret detection libraries with very different capabilities. For example, Yelp's detect-secrets can only scan the current source code without the git histories and rely on an interactive way to reduce the false positive. Picking one that's easy to use is a challenge.
  2. TruffleHog is a nice library that only has ~400 lines of the code. One of its options --since-commit is broken and I had to find a patch for that.
  3. As of now, demisto-sdk has a bug in uploading the new layout container JSON.

Accomplishments that I'm proud of

I think this integration is unique and agile. It's easy to set it up, and we already have incidents created for the SOC team. The team can also make it better by tuning the patterns.

What I learned

This is actually the second integration I have worked on, hence I have not learned a lot in the XSOAR area. Doing this project is purely for the "fun" and my team, Palo Alto Networks IT First Customer, is encouraging me to do so.

What's next for GitHub Secrets Detection in XSOAR

  1. We should add some unit tests.
  2. Besides doing secret detection, static code analysis can also be integrated.

Demo Video

That picture of my daughter Johanna and me was taken 6 years ago. I found it funny we were working on our "laptops" together and now she is already helping me with the demo!

Built With

Share this project:

Updates