Detecting Lateral Movement activity: A new Kibana integration

Cyber attacks are becoming more frequent, targeted, and complex. When it comes to sophisticated attacks, one of the most commonly seen tactics is Lateral Movement. During lateral movement, many attackers try impersonating a legitimate user by abusing admin tools (e.g., SMB, SAMBA, FTP, WMI, WinRM, and PowerShell Remoting) to move laterally from system to system in search of sensitive information. In addition, they may avoid using traditional malware to stay beneath the security radar, making detection more difficult.

This publication will discuss the newly available Lateral Movement Detection package under Kibana’s integrations app. Our new package includes anomaly detection jobs and security rules and takes advantage of the endpoint malware classifier to make finding Lateral Movement in your network simpler.

Brief introduction to Lateral Movement

Lateral movement is a class of techniques used by adversaries to access resources in an enterprise that may store valuable data or sensitive information. For the purposes of our package, it also pertains to remote execution events that leverage the same protocols. These techniques play an essential role in the mobility of attackers who have established a foothold in the network and need to access one or more other systems until they achieve their ultimate objective(s). This may include accessing or destroying sensitive files and running a malicious payload on the target system.

Intrusions can escalate quickly as adversaries attempt to outmaneuver security technologies and analysts. Preventative measures such as enterprise controls (segmentation), endpoint capabilities (sensors with detection capabilities), and user entity behavior analytics (UEBA) exist, but none of these can address all the possible use cases alone.

Our approach

Typically, attackers rely on built-in protocols to deploy malicious software on victim hosts or to move data outside the enterprise network. Evaluating the use of these protocols is a key aspect of how Elastic escalates these behaviors to your attention.

Our new Lateral Movement Detection framework complements behavioral and endpoint detections to provide mitigations against malicious software. In addition, several anomaly detection jobs are employed to profile host-to-host file transfer behavior. To monitor file transfers at the endpoint, we look for file “creation” events with a “process.name” of “smbd,” “system,” “vsftpd,” “sftp-server,” “sshd,” or “scp.” The aforementioned “process.name” values correspond to the five most commonly used file-transfer protocols for Windows and Linux hosts (e.g., SAMBA, SMB/PS Remoting, FTP, SFTP, SSH, and SCP). For this package, we decided to start with these services first with the aim of expanding the capability in future releases.

The table below demonstrates a host to host matrix with their respective services:

We filter file creation events based on the above table and use file metadata such as file size, extension, and directory to profile hosts for abnormal file transfer activities using anomaly detection jobs. These jobs flag potentially malicious activity by looking for spikes in file creation events, large amounts of data transfer, and rare file extensions and directories. Such outliers in the file transfer activities and metadata could indicate compromise, and may suggest that someone is trying to move malicious content across the network.

The following flowchart depicts the Lateral Movement Detection package at work (note numbers are examples for illustrative purposes)

Moreover, our package also uses detection rules and endpoint features like the MalwareScore classifier — mixing and matching features to add confidence. For instance, we look for file “creation” and “modification” in sensitive directories, (e.g., admin shares for windows, dot, config, and bin files for Linux). Adversaries may attempt to create or modify files in folders like “bin,” where they can execute a malicious binary hidden among legitimate ones. In addition, we use the correlation between remote file creation events and MalwareScore detections to identify a malicious file transfer, one of the most common ways malware propagates in an enterprise. Our model is trained on a diverse feature set, allowing it to learn all the patterns in data pointing toward a malicious file. Finding and implementing these patterns using behavioral logic is tedious and could result in more noise.

Getting started

As of 8.6, users can install the Lateral Movement Detection package from the Integrations app in Kibana. If you don’t have an Elastic Cloud cluster but would like to start experimenting with the package, you can start a free 14-day trial of Elastic Cloud.

To set up and begin using the Lateral Movement Detection package, follow the steps listed below.

Step 1: Installing the package

In the Kibana integration app, the new Lateral Movement Detection package is available. To install, go to the Settings tab and click on Install Lateral Movement Detection assets. This will install all the artifacts under this package to generate events and alerts when Lateral Movement is detected in the file transfer activity.

Step 2: Running anomaly detection jobs

As discussed above, this package consists of several prebuilt anomaly detection jobs. To run these jobs, go to Machine Learning > Anomaly Detection. Go to the Jobs wizard under Anomaly Detection and click on Create job.

Select the Lateral Movement Detection card under the Use preconfigured jobs option and click on the Create jobs button.

Step 3: Enabling security detection rules

This package uses anomaly detection jobs, security rules, and the endpoint malware classifier to detect lateral movement in the network. To see all the Lateral Movement Detection alerts, you need to enable all the installed detection rules under the package. The rules based on anomaly detection jobs are triggered when the anomaly score surpasses a predetermined threshold which can be customized by duplicating the detection rule.

To enable and use the installed rules, navigate to Security > Alerts > Manage rules and select Load Elastic prebuild rules and timeline templates. If you’re running in Elastic Cloud, this step is done automatically.

You can use the Lateral Movement Detection tag to filter for all the detection rules under this package.

To enable a specific rule, click on the Enabled toggle corresponding to the rule name. If you want to enable all the rules, select the rules and click on Enable option under the Bulk actions.

Responding to Lateral Movement Detection alerts

You can track the anomalies triggered by the anomaly detection jobs under Anomaly Detection > Anomaly Explorer.

Note that you will only see a security alert for these jobs if their Severity surpasses the Anomaly score threshold assigned to them in the detection rules. For example, the rule “Spike in Remote File Transfers” assigns a threshold of 70 to the anomaly detection job “high-count-remote-file-transfer.” This job is given a higher threshold to differentiate it from the baseline or normal remote transfer activities in the network.

Furthermore, you can also perform the below changes to the rules depending on your local environment. You would have to duplicate the rule to edit its settings.

1. Change the Severity and Risk score of the rules if you want to subdue or uplift their importance in your network (a detection rule with a higher score means it is more severe and risky).
2. Change the Anomaly score threshold for the rules to trigger alerts at different thresholds.

Get in touch

We’d love for you to try out the Lateral Movement Detection package and give us feedback as we work on adding new capabilities to it. If you run into any issues during the process, please reach out to us on our community Slack channel, discussion forums, or our open detections repository.