Necessity is the mother of invention. Threat hunting and investigation requires a point of pivot and a view that encompasses situational awareness around that pivot. This provides analysts with the critical context to analyse an incident scenario across a timeline and all security controls. There are mainly 3 central pivots, user, ip, host machine. We could not find a way in Sentinel to have that pivot based on a user during our investigations so we decided to build it. The user pivot is extremely useful in identifying the cases of account theft/infiltration/admin compromise/initial entry. Currently within the Sentinel investigation panel you can only pivot on a user with scope limited to sentinel alerts and details limited to very basic information. Any further details on the user pivot can only be seen as a query output which is not visually insightful.

What it does

This product provides a workbook to quickly investigate user activity across all security tools feeding into sentinel and contextualize the investigation to aid the new pivot point. It utilises the strength of workbook visualisations to create a view that an analyst can interpret easily and succinctly.

How I built it

In order to demonstrate the benefits an array of applicable sources were utilised e.g proxy, cloud app, signin, office activity etc. We onboarded custom log sources such as ZScalar as well Microsoft security technologies and then created KQL queries with custom extractions to build the workbook panels. The logs from these various security controls are typically analysed when ascertaining the scope of an attack via a compromised account. The various log sources have different requirements to satisfy the condition of identifying the account associated to an activity. Normalisation is still a largely unsatisfied feature within Sentinel. KQL and REGEX is used for the differing extraction of the user account details in the various sources. The panels – each addressing a different source - within the workbook, are used to bring all the data for that user in one view. Supporting platforms are also linked within the workbook to support the analyst’s view of that user e.g AAD user profile. Some data sources such as O365 Security Alerts would not consistently have user account details in all events.

Challenges I ran into

No existing way to deep link a workbook from the incident to auto-fill the input values and many time the extractions had to be done in KQL. Not all alerts have user account details within the logs so these type of alerts cannot be connected into the user view. Different sources store different user data e.g username vs email vs first and last name (johnd1,, john doe). To link these all to the same user - AAD data needs to be used and pulled into Sentinel, stored in a table that can be referenced.

Accomplishments that I'm proud of

The workbook has been very useful in real world scenarios already and has aided our analysts to perform their investigations faster. The workbook is very directed, it manages to provide great context without being noisy, and gives much better guidance for a hunt. It also provides much better situational awareness to guide decision making in terms of what hunt avenues to pursue.

What I learned

Extraction from json and string fields. The flexibility of KQL, KQL querying. Interactive workbook creation, e.g showing a panel based on parameter conditions. The various strengths of workbook parameters. How to create graphs, interactive tiles, tabs, maps, clickable grids. These features available within workbooks being fully realised to facilitate effective and useful data analysis.

What's next for Sherlock User Investigator

We would like to expand Sherlock to show a comparison for the user against the rest of the organization or their peer group which can be used as a trigger. Furthermore add the ability to trigger playbooks from within the workbook via custom actions. Additionally, we plan to make this longer term a “one hunt to rule them all” workbook by including in-built queries and pivots for Machines, IP addresses and IOCs (e.g. Hashes, filenames, processes).

Built With

+ 41 more
Share this project: