Microsoft Sentinel Machine Learning (ML) based Threat Detection Rules
By: Zubair Rahim
In Microsoft Security Insight Podcast Innocent Wafula and Rod Trent talk about the ML capabilities of Microsoft Sentinel. In this write-up, I am focusing on the threat detection rules based on ML.
1. What is Built-in ML Rule?
2. How to Enable Built-in ML rule
3. What is a Customizable ML rule?
4. How to create and customize Customizable ML rule
5. How to trigger an alert when Anomaly is detected.
6. Bring your own Machine Learning Platform
Built-in ML Threat Detection Rules:
ML behavioral analytics templates are based on proprietary Microsoft machine learning algorithms, so you cannot see the internal logic of how they work and when they run.
Because the logic is hidden and therefore not customizable, you can only create one rule with each template of this type.
To create/ enable these rules go to Analytics > Rule templates. Click on Rule Type and select Fusion and ML Behavior Analytics.
To enable ML Behavior rule select rule and click on Create rule after that click on Next: Automated Response > if you want to configure Automation Response if not click on Next: Review and click on Create. Allow 7 days after this alert is enabled for Microsoft Sentinel to build a profile of normal activity for your environment. If these rules are not available in your environment so ping me on LinkedIn.
When the alert triggers, it will automatically create an incident.
Fusion: Microsoft Sentinel uses the Fusion correlation engine, with its scalable machine learning algorithms, to detect advanced multistage attacks by correlating many low-fidelity alerts and events across multiple products into high-fidelity and actionable incidents. Because the logic is hidden and therefore not customizable, you can only create one rule with this template. In Active Rule select, Advanced Multistage Attack Detection is enabled by default. Make sure that all the Data connectors are enabled in the rule.
Customizable ML Rule:
With attackers and defenders constantly fighting for advantage in the cybersecurity arms race, attackers are always finding ways to evade detection. Inevitably, though, attacks will still result in unusual behavior in the systems being attacked. Microsoft Sentinel’s customizable, machine learning-based anomalies can identify this behavior with analytics rule templates that can be put to work right out of the box. While anomalies don’t necessarily indicate malicious or even suspicious behavior by themselves, they can be used to improve detections, investigations, and threat hunting.
Some of the Anomalies are enabled by default but you can enable or disable the anomalies according to your data sources. To Enable Anomaly, go to Analytics > Rule templates. Click on Rule Type and select Anomaly you can also filter Anomaly by selecting Data sources.
As I said these rules are customizable up to some extend so to customize these Rules In the Active Rules tab Right-click on Anomaly select Duplicate.
So, the duplicate rule will be created with suffix Customized.
Now click on edit after that select Mode Production or Flighting. If you select the Production Mode the original Anomaly will be changed to Flighting mode. Click next on Configuration now you can change the Anomaly Score Threshold. The Anomaly Score threshold is a score to generate an anomaly when the anomaly score is greater than the chosen value.
The Anomaly does not trigger the Alert you can find that Anomalies when triggered in Anomalies table.
Now if you want to trigger Alert when Anomaly is detected so you can create a schedule Rule or NRT rule. To create an NRT query rule go to Analytics click on create and select NRT query rule
In the Rule Query write the below Query
Anomalies
| project TimeGenerated, AnomalyTemplateName, Description, UserName, RuleConfigVersion, Score, ExtendedLinks, Tactics, Techniques
Enable or Disable the Incident.
In Alert Automation Select Playbook.
Then click Next: Review after validation passed Create the rule.
Here You can see when the Anomaly detected in My Environment triggered the Alert.
Bring Your Own Machine Learning (BYO-ML) platform:
For organizations that have ML resources and would like to build customized ML models for their unique business needs, Microsoft Sentinel offer the BYO-ML platform. The platform makes use of the Azure Databricks/Apache Spark environment and Jupyter Notebooks to produce the ML environment.
For More Details:
https://docs.microsoft.com/en-us/azure/sentinel/bring-your-own-ml
Thank You
