A game of nations — How nation-states prepare for cyber threats with Locked Shields (part 2)

How Elastic Security is used in Locked Shields; one of the worlds largest cyber defense exercises.

center-of-excellence-2.jpg

This blog is part 2 of a previous blog, which introduced Locked Shields. It gives a brief overview of how Elastic® is typically set up during Locked Shields. There is also a focus on the meticulously executed comprehensive series of attack simulations on systems housing critical infrastructure in the fictitious country of Berylia. 

Within the controlled Locked Shields environment, Elastic SIEM and Endpoint Security demonstrated their capabilities by successfully detecting and preventing a myriad of attack tactics and techniques. The blog encapsulates a brief summary of the top incidents that emerged during these simulations, offering valuable insights into the efficacy of Elastic Security as a unified cybersecurity solution. 

While the blog does not go into the specifics, high-level threat trends will be shared to show how Locked Shields is as realistic as a cybersecurity exercise can get.

Left and right of boom in cybersecurity

Most teams at Locked Shields take the approach of left-and-right side of boom. The concept of "left" and "right" side of boom in cybersecurity refers to different aspects of the cybersecurity landscape.

1 - boom
An example of considerations before, during, and after boom*

The "left" side generally alludes to the focus on prevention strategies to stop threats before they manifest. It encompasses strategies such as security architecture design, implementation of firewalls, intrusion detection/prevention systems, access controls, encryption, and other preemptive measures. Activities on the left side focus on the proactive securing of systems and networks to prevent unauthorized access, data breaches, malware infections, and other cyber threats.

On the other hand, the "right" side pertains to reactive measures taken after a cybersecurity incident or breach has occurred. In this stage, it is cardinal to have a well-prepared incident response plan. The plan should outline the steps to identify, contain, and analyze the security incident. The focus is on mitigating the impact of the cyber attack, identifying the root cause, restoring systems to normal operation, and implementing measures to prevent similar incidents in the future.

Learn more in our on-demand webinar: Left and right of boom in cybersecurity

The setup and configuration

So how did teams apply the left and right of boom principles on Elastic? Once assets housing critical applications and data were identified, the next step was to collect relevant audit data from them. Member nations make use of Elastic's out-of-the-box integrations to make data ingestion and connecting to other data sources easy — allowing them to store, search, and analyze the data.

The power of Elastic Common Schema

The integrations parse and normalize the data into the Elastic Common Schema (ECS), and most ship with out-of-the-box dashboards, making security engineers’ tasks easier. The parsing and normalization is a critical step to make detection and analytic workflows streamlined across the common fields for disparate data sources. 

For Locked Shields, the Elastic environment was thoughtfully configured, featuring the installation of Elastic Agent on various systems, complemented by a range of critical integrations geared toward monitoring and protection. 

Data sources:

  • OS event logs
  • PowerShell logs
  • Firewall and IPS/IDS logs
  • Threat intelligence
  • Elastic Defend integration (Elastic’s EDR offering) 
    • All event collection enabled for full endpoint telemetry visibility
    • Set to Prevent, so anything detected should’ve been stopped
  • Osquery Manager
  • File integrity monitoring, among others

To prepare effectively for "boom,” these data sources were leveraged and aligned with various techniques outlined in the MITRE ATT&CK® framework. For example, PowerShell logs and file integrity monitoring were used to detect execution and persistence techniques, while OS event logs and endpoint logs were leveraged for detection rules for privilege escalation activities. Detection of defense evasion tactics was facilitated by firewall and IDS/IPS logs, along with endpoint logs and Osquery Manager for insight into evasion attempts.

By correlating data from these sources with MITRE ATT&CK tactics and techniques, the teams enhanced their threat detection and response capabilities across the entire attack anatomy, from reconnaissance to the explosion of an incident (“boom”).

Gain the upper hand over adversaries with osquery

Security analysts made use of osquery, a tool that allows querying operating systems like a database using SQL. Osquery is deployed to all agents using the osquery manager integration to strengthen XDR capabilities. Analysts were able to run live queries and schedule queries to gather data from hundreds of tables across the endpoint assets. 

For example, an analyst can quickly search the process tables across related hosts for a specific running process or search for malicious cron jobs via the crontab table. Osquery allows analysts to get more context about hosts and assets to help inform an investigation.

2 - An example of an osquery search for malicious crontab commands on Linux
An example of an osquery search for malicious crontab commands on Linux

Osquery is the perfect complement to real-time streaming data sources on the Elastic Stack. It allows for parallel static searching, all unified in a single, normalized search and alerting ecosystem. This helped with real-time incident investigations, threat hunting of red team activity, and regular monitoring to detect suspicious activity on the endpoints. See this walkthrough for endpoint monitoring with osquery and Elastic and this blog on how to threat hunt for persistence with osquery. 

Anti-malware protection with Elastic Defend

For Locked Shields, the blue teams were able to deploy the Elastic Defend integration via the Elastic agent, including detection sensors and protection packages. The integration allows for enhanced visibility into events occurring on Windows, macOS, and Linux operating systems that enable the security analytics and rules. The events and telemetry collected by Elastic Defend are streamed to the Elastic Stack for further analysis and detection workflows. 

In addition, the Elastic Defend integration includes the core protection technology. This technology prevents malware and ransomware from executing and stops advanced threats using:

  • Malicious behavior analysis

  • Memory threat protection

  • Credential hardening protections

All these capabilities are powered by Elastic Security Labs and our global community. See our protections-artifacts repo for more info on the protection capabilities of Elastic Defend.

Holistic visibility and correlation with Elastic SIEM

To provide 360-degree visibility into the environment and to correlate the disparate data sources in Berylia, the teams utilized Elastic SIEM (security information and event management). Elastic SIEM is a comprehensive security solution designed to help organizations detect, investigate, and respond to security threats across their digital environments. The teams primarily used the SIEM for:

  • Threat detection 

  • Alerting and notification

  • Threat intelligence integration

  • Correlation and contextualization

  • Incident investigation and response

The ease of getting started with prebuilt detection rules makes the adoption of Elastic SIEM easy. Users were able to customize the prebuilt rules where needed and also create rules from scratch for custom threat use cases. Context was added to the rules through use of exception lists, when trusted users needed to be added as an exception to a rule, for example.

With all assets identified, Elastic agents installed to collect data from the assets, endpoint protection turned on with Elastic Defend, and the SIEM detection rules enabled and tuned accordingly, the teams were ready to thwart various attacks from the red teamers.

Live fire days: What did we see?

Anatomy of an attack

Before going into discussing the live fire days, it is imperative to understand the anatomy of a cyber attack. A framework that helps with this is the previously mentioned MITRE ATT&CK framework. MITRE ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge) is a framework for understanding attacker behaviors and techniques. It provides a structured taxonomy of tactics and techniques that attackers use during cyber attacks. 

Each tactic within the MITRE ATT&CK framework can be treated as a stage in the attack lifecycle. Understanding how these tactics show an attack anatomy helps defenders anticipate, detect, and respond to cyber threats effectively.

3 - An attack anatomy from a MITRE ATT&CK perspective
An attack anatomy from a MITRE ATT&CK perspective

Elastic’s SIEM detection rules are mapped to the MITRE ATT&CK framework, and so too are the resulting alerts across Elastic Security. Many mature security teams look to the MITRE ATT&CK matrix to help improve their understanding of attacker tactics, techniques, and procedures (TTPs) and to better understand their own capabilities relative to these common adversarial approaches. Elastic Security has embraced this matrix for a while now and continues to innovate around it

Defense Evasion: Most prevalent tactic

An interesting observation is that Defense Evasion was the most popular tactic overall during Locked Shields, and that might not necessarily be so surprising. Red teamers use various techniques to avoid detection or hinder the effectiveness of defensive measures employed by defenders. They employ a variety of methods to evade detection, and the ATT&CK Defense Evasion tactic covers these techniques.

4 - A grouping of tactics and techniques in the alerts during the entire Locked Shields
A grouping of tactics and techniques in the alerts during the entire Locked Shields

The most notable and interesting techniques used for defense evasion were the following: 

  • Indicator removal 

  • Impair defenses

  • Command and scripting interpreter

  • Process injection, among others

Indicator removal
Attackers continuously attempted to clear or modify log files on compromised systems to remove traces of their activities. This included event logs, system logs, or any other audit trails that could contain evidence of their presence.

5 - An example rule for detecting indicator removal on a windows host
An example rule for detecting indicator removal on a windows host

Impairing defenses
This one is an interesting technique. Red teamers aimed to undermine the effectiveness of defensive strategies employed by the blue teamers, with the goal to make it easier for them to achieve their objectives. Some of the actions taken involved hindering or disabling security tools or mechanisms implemented by defending teams to protect their systems and networks. 

To add in a realistic aspect to Locked Shields, red teamers attempted to impair endpoint protection by proactively uninstalling the Elastic Agent through hard coded commands, or after doing a privilege escalation on endpoints. This was upon discovering that some teams were using Elastic agents on their assets. The release of enhanced agent tamper protection couldn’t have been timelier for such scenarios! When enabled, Elastic Agent and Elastic Endpoint protection can only be uninstalled on the host by including an uninstall token in the uninstall CLI command. 

Command and scripting interpreter
This technique was equally prevalent. It's a classic scenario of functionality and ease of doing things (automation) versus security. Scripts and use of the shell make a lot of IT workflows easier by enabling automation; however, attackers also leverage these capabilities for malicious intent. 

Red teamers took advantage of these technologies in various ways as a vector of executing arbitrary commands. Commands and scripts were seen being used in initial phases of attacks, for example in Initial Access or as secondary payloads downloaded from an existing command and control center (c2). Red teamers also executed commands through interactive terminals/shells, as well as many other Remote Services in order to achieve execution of malicious code remotely.

Prevalence of “living-off-the land”

Attackers have a tendency to leverage legitimate scripts, binaries, or libraries already present in the target environment to carry out malicious activities, instead of solely relying on malware or external tools. This approach is called "living off the land" (or LOL in short), and we saw a lot of it during Locked Shields. LOL techniques fall under various tactics in the ATT&CK framework. Attackers leveraged various LOL techniques covered under Defense Evasion, Persistence, and Execution ATT&CK tactics, among others.

Examples of common entities that are used for “living off the land” include:

  • PowerShell

  • Windows Management Instrumentation (WMI)

  • Remote Desktop Protocol (RDP)

  • Scripting languages

There is a project called LOLOL that aggregates LOL approaches and projects. The above examples are Windows utilities only. The complete list is tracked in a project called LOLBAS. There is an equivalent Linux project called GTFOBins that keeps track of similar legitimate tools commonly exploited by attackers. All these and other LOL projects can be tracked via the LOLOL project.

The “living off the land” approach is favored by adversaries because it helps them evade detection by blending in with normal system activity and leveraging trusted processes and tools, making it more challenging for defenders to distinguish malicious behavior from legitimate actions. 

To circumvent this, Elastic Security monitors for anomalous behavior of authorized tools and utilities, and once the confidence of malicious intent is established, the suspicious activity is flagged.

Memory-based threats

Similar to “living off the land” activity, the red teamers utilized various memory and kernel level attack techniques, as could be seen in some of the alerts picked up by Elastic Security. A lot of anti-malware tools can today easily detect malicious file signatures on endpoints. As a result, attackers tend to opt for attack vectors that don’t require writing any files to disk, so as to evade EDR and AV tools. 

Aiming to inject payloads in processes running in memory, or processes assigned memory space, is common. This technique is called in-memory injection. 

Elastic Defend was able to prevent memory manipulation via shellcode, stopping sub-techniques such as thread execution hijacking, asynchronous procedure call, process hollowing, and process doppelgänging, among others. Elastic Defend also utilizes in-memory yara signatures as described in the protections-artifacts repo.

This protection technology provided security teams with another layer of prevention against attacks engineered to evade less advanced endpoint security technologies.

Discovery: Crucial tactic in the attack anatomy

It is no secret that adversaries are known for their patience when carrying out attacks. Initial access is usually achieved on non-critical assets, so once attackers are in the environment they must move laterally to discover crown jewels. This is where the Discovery phase comes in. Patience is key! They understand that any rush could lead to increased oversight or worse still detection. 

The figure below is an exhibition of the prevalent tactics in the first half of the live fire days of Locked Shields. It does affirm that red teamers spent a good chunk of time on discovery once they got persistent and escalated access to assets.

6 - A distribution of tactics halfway through Locked Shields; Discovery was the most prevalent tactic at this stage
A distribution of tactics halfway through Locked Shields; Discovery was the most prevalent tactic at this stage

By patiently knitting together seemingly inconsequential pieces of information during the Discovery stage, adversaries construct a comprehensive profile of their target's infrastructure, vulnerabilities, and likely lateral movement points. Both manual and automated strategies are employed in this stage. 

This methodical approach allowed red teamers to exploit vulnerabilities effectively and launch more clear-cut and malicious cyber attacks, emphasizing the significance of patience as a strategy in cyber warfare.

Key takeaways: Is Locked Shields a realistic exercise?

Attack anatomy similar to real-life attacks

The impressive thing with Locked Shields is that the stage is always set with a theme year-in, year-out. The story is always centered on the two fictitious countries: the aggressive Crimsonia nation-state launching cyber warfare on Berylia. Given the MITRE ATT&CK trends seen above, it is easy to see that red teamers take a well structured approach to how they execute their attacks. The objective of attack for each asset of Berylia is always well established and is preceded with initial access landing activities as well as escalate, discovery, and other tactics.

7 - Structured steps an adversary takes to execute an attack
Structured steps an adversary takes to execute an attack

Very targeted payloads

Now, it is worth noting that Locked Shields has rules of engagement for both the offensive and defensive teams. However, they also try to strike a balance between being a competition and having that realistic aspect to them. Hence, it isn’t surprising to see red teams take actions informed by reconnaissance or any other information gathering steps. A good example is the leverage of backdoors in Berylia’s systems.  

As already highlighted, there were some traces of process arguments that had explicit commands to uninstall the Elastic Agent in the data. Use of backdoors, privilege escalations, and other techniques and customized strategies enforce the reality that defenders need to expect when encountering aggressive threat actors in real life.  

Elastic was not the only defense capability targeted — other tools in the blue team's arsenal were also targeted, as shown in the alert trends. Again, this affirms that Locked Shields trains defenders to expect this level of aggression in real life.

The user is often considered the weakest link in cybersecurity due to the inherent challenges associated with erratic human behavior. This was evidenced during Locked Shields. Despite advancements in technology and robust security measures, individuals can compromise the integrity of a system through actions such as using default credentials, weak passwords, or neglecting security best practices. 

Locked Shields shows that even well-designed security infrastructures can be undermined if users are not adequately trained and aware of potential threats. Therefore, organizations must prioritize user education, implement stringent access controls, and foster a culture of cybersecurity awareness to strengthen this critical link and mitigate the risks associated with human factors in the ever-evolving landscape of cyber threats. The left side of boom is indeed a crucial step in cyber risk management.

The advent of technology that focuses on posture management can also mitigate some of the human errors, such as misconfigurations and use of vulnerable software libraries. For example, Elastic’s cloud security posture management (CSPM) and its related Kubernetes counterpart KSPM have been designed for this purpose. This is extended with Cloud Native Vulnerability Management (CNVM), which helps with identifying known vulnerabilities in cloud workloads.

Collaboration is key to combating cyber threats

Locked Shields, in addition to technical submissions, scores on how well teams collaborate and how they share information. The most successful teams are those who manage to tackle all challenges in different categories. There is a balance between strategic decision-makers and technicians who all have to work together to properly address all the elements of a large-scale cyber attack**.

Overall, exercises such as Locked Shields play an important role in spreading awareness about the reality of nation-state cyber attacks. The emphasis on process and people readiness, in addition to use of security protective technologies, is an invaluable part of Locked Shields. May the games continue! 

Get started

Elastic Security protects organizations and their data from attack. The solution can thwart complex attacks with advanced analytics, blocking threats such as malware and ransomware on critical systems. Elastic Security enables the advancement of security operations maturity to stop threats at scale. It is simple and easy to get started with Elastic Security.

References
* https://www.balbix.com/blog/left-of-boom-firemen-cyber-security-radar-and-breach-risk
** https://ccdcoe.org/locked-shields

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.