This first post of two motivates the "why" and explores the "how" behind malware prevention via supervised machine learning in Elastic Endpoint Security.

<p>We recently launched Elastic Security, combining the threat hunting and analytics tools from <a href="/products/siem">Elastic SIEM</a> with the prevention and response features of <a href="/products/endpoint-security">Elastic Endpoint Security</a>. This combined solution focuses on detecting and flexibly responding to security threats, with machine learning providing core capabilities for real-time protections, detections, and interactive hunting. But why are machine learning tools so important in information security? How is machine learning being applied? In this first of a two-part blog series, we’ll motivate the "why" and explore the "how," highlighting malware prevention via supervised machine learning in Elastic Endpoint Security. In a follow-up, we’ll highlight where machine learning plays a role in detecting anomalous behavior from the global view of Elastic SIEM.
</p> <strong>
<h2>Malicious activity: a case study</h2></strong>
<p>Even as the United States prepares for the 2020 election cycle, a three-year anniversary of an <a href="https://www.cnet.com/news/how-the-russian-hackers-infiltrated-the-dnc-a-timeline/">infamous cyber attack</a> is still top of mind.
</p>
<p>On Tuesday, March 15, 2016, threat actors allegedly from the advanced persistent threat (APT) Group 28 (aka “Fancy Bear”) began port scanning servers at the Democratic National Committee. This was part of a broader reconnaissance effort that adversaries commonly perform against a target. By Saturday, the adversary had delivered a spear-phishing email to the campaign chairman to gain access to his email account. Although recipients noted it as suspicious, the attack worked: by Monday, the threat group had compromised the chairman’s email account and stolen more than 50 thousand emails. Then, from a new email account created by attackers within the DNC domain (with a name similar to the account of a recognized DNC staffer), a spreadsheet named “candidate favorability ratings” was delivered to 30 DNC staffers. At least one recipient opened the spreadsheet and enabled a macro, which resulted in a local user account being hacked.
</p>
<p>In the weeks that followed, a common playbook of tactics unfolded in what is now eponymously known as the DNC hack. Like other APT groups, while the specific techniques deployed are circumstantially tailored, the broader tactics are repeated in attack after attack (and thus codified by MITRE). The attack process typically follows some variation of the following steps:
</p>
<ul>
	<li>Human exploitation leads to account theft.</li>
	<li>Attackers establish persistence, use existing tools or install command-and-control malware (in the case of the DNC hack, X-Agent) that allow them to log keystrokes, take screenshots, and otherwise “land and expand” into other servers on the network.</li>
	<li>Attackers collect, compress, and consume a surfeit of data.</li>
	<li>Logs are scrubbed and digital footprints are delicately swept away.</li>
	<li>The threat group goes dormant, becoming difficult to fully root out, even if their presence is detected on a single device (in the case of the DNC hack, although X-Agent was disabled on June 20 — <strong>97 days after the initial reconnaissance</strong> — a Linux version of the malware remained through October).</li>
</ul>
<p>It’s not difficult to understand why unified protections and comprehensive event management are important in a case like this. Unfortunately, being compromised has become a fact of life, and security organizations aim to stop an attacker early in the execution of their plan. Thus, most of the energy traditionally spent in security has been toward detection mechanisms that reduce the dwell time of an attacker post-compromise. Since attacker tools and techniques vary, rules crafted for specific adversary techniques and machine learning trained to catch more general behaviors have been used to detect almost every point along the attacker’s natural progression from reconnaissance to delivery; exploitation to exfiltration.
</p>
<p>The <a href="https://attack.mitre.org/">MITRE Corporation</a> began developing a globally accessible knowledge base of adversary tactics and techniques (ATT&CK) based on real-world observations in 2014, which has now grown to codify 266 specific techniques, including many created or enhanced by community <a href="https://www.mitre.org/capabilities/cybersecurity/overview/cybersecurity-blog/the-philosophy-of-attck">contributions of information</a>. The MITRE ATT&CK matrix is now the most comprehensive adversary model and framework for describing the actions an adversary may take to compromise and operate within an enterprise network.
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltfbac7da809939393/5ddc4ca316efc60c370c15af/ml-cybersecurity-mitre-framework-blog.jpg" data-sys-asset-uid="bltfbac7da809939393" alt="Mitre Attack matrix" style="display: block; margin: auto;">
</p>
<p><a href="https://attack.mitre.org/"></a>
</p>
<p><strong></strong>
</p>
<h2><strong>Why machine learning?</strong></h2>
<p>Machine learning offers an attractive paradigm for scaling the detection of adversarial behaviors and their tools, largely owing to the following:
</p>
<ul> <strong>
	<li><strong>Automation</strong> — Coarsely, security has historically been tackled by throwing experts at data to produce rules, heuristics, and analytics in a repetitive process of cat-and-mouse. One natural outcome of machine learning is the repurposing of security experts to engineer automation processes — including machine learning training infrastructures — rather than spending cycles on the most repetitive tasks of detection creation. In this way, it’s the algorithms, not valued experts, that are tackling the mundane tasks, occasionally overseen, curated, and refined by security experts where appropriate.</li></strong> <strong>
	<li><strong>Discovery of sophisticated relationships</strong> — Human experts are still best positioned to pinpoint causal indicators of threat behavior. For example, a reverse engineer may disassemble and isolate a particular code block encapsulating nefarious behavior. However, machine learning is exceedingly good at finding useful correlations from data that can’t easily be discovered manually. For example, some combination of file size, imported functions, resource table usage, and properties of the <code>.text</code> section might isolate a 99.99% pure set of malicious files in the training data. While this isn’t a smoking gun for malicious behavior, it may offer enough evidence to convict a file, process, or user of malintent.</li></strong> <strong>
	<li><strong>Generalization with less brittleness</strong> — The discovered patterns tend to generalize toward new malware strains, similar tactics, or sketchy user behavior in a way that is less brittle than hand-crafted rules. Machine learning models are often shaped in a way that a file, process, or behavior is not convicted as a binary “good” or “bad,” but instead produces a score that smoothly transitions between areas convicted as “good” and areas convicted as “bad.” Importantly, a model is generated from the data. All real-world machine learning models are in some sense “<strong><a href="https://en.wikipedia.org/wiki/All_models_are_wrong">wrong</a></strong>,” in that they do not perfectly summarize their training data. However, they are “useful” in their very specific task of rendering judgements on samples that are similar to those in their training set. They have “learned” to <em>generalize.</em></li></strong>
</ul>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blta0f4392313f43879/5ddc4cead800b20c30e1ab85/ml-cybersecurity-space-code-blog.jpg" data-sys-asset-uid="blta0f4392313f43879" alt="Malware mapping 1" style="display: block; margin: auto;">
</p>
<p><br>
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt4fdf6710c32a32da/5ddc4d07d800b20c30e1ab8b/ml-cybersecurity-space-size-blog.jpg" data-sys-asset-uid="blt4fdf6710c32a32da" alt="Malware mapping 2" style="display: block; margin: auto;">
</p>
<h3>No free security lunch</h3>
<p>Marketing campaigns notwithstanding, machine learning has generally been a benefit to security. But, the truth is that machine learning for security brings a number of substantive challenges:
</p>
<ul> <strong>
	<li><strong>False positives</strong> — When compared to hand-crafted rules and heuristics, machine learning generally trades an increased “true positive rate” (detect new behavior) for a modest increase in false positives (FPs). While the FP rate is still less than, say, 1 in 2000, this is discouraging when the model chooses <em>your </em>favorite application to flag as malicious. “Fixing” a FP is not straightforward. To borrow a <strong><a href="https://pbs.twimg.com/media/EHHdPcCXUAABs3d.jpg">concept</a></strong> from Microsoft, the challenge of deploying machine learning is that “one doesn’t so much fix errors in ML as much as encourage toward the correct answer.” In fatalistic irony, FPs actually <em>increase</em> analyst workload, reducing the overall effectiveness of a solution, especially when defenders respond to FPs by the <strong><a href="https://www.reuters.com/article/us-target-breach/target-says-it-declined-to-act-on-early-alert-of-cyber-breach-idUSBREA2C14F20140313">age-old approach</a></strong> of ignoring or turning off alerting.</li></strong> <strong>
	<li><strong>Explainability</strong> — In many cases, from a human’s perspective, the results of a machine learning model may seem incoherent or inconsistent. Trust is paramount in security, and the explainability of a model may be at odds with its sophistication and accuracy. Indeed, when offered, some explanations of model decision come across as trend diagnosis, e.g., “this file is malicious because of imports and high entropy in <code>.text,</code>” which does not provide a true explanation, but instead reinforces a notion that machine learning amounts to “bias automation.”</li></strong> <strong>
	<li><strong>Adversarial drift</strong> — Dataset drift in machine learning is present in many applications due to the natural evolution of observations. In security, where an adversary — a <em>who</em> not a <em>what</em> — is the focus of observed data, the drift is driven in an adversarial way: an adversary is incentivized to adjust behaviors to evade defensive detections. In this scenario, the fundamental assumption of machine learning — that the training conditions of a model are identical to deployment conditions — is broken. Instead, this assumption must be relaxed to “deployment conditions are almost identical to training conditions for some period of time [until they aren’t].” To keep up, defenders must recollect, retrain, and redeploy machine learning models, typically at a much slower pace than brittle signature and rule-based approaches.</li></strong>
</ul>
<p>Tackling these and other considerations motivate the need for security data science and engineering at Elastic. So, where have we applied machine learning? Here, we’ll describe where machine learning plays a role in Elastic Endpoint Security protections. In a follow-up, we’ll highlight where it plays a role in detecting and searching from bulk event data in Elastic SIEM.
</p> <strong>
<h2>Endpoint security protections: MalwareScore™</h2></strong>
<p>Both custom and common software tools are employed by threat actors across all tactics in the MITRE ATT&CK framework. In particular, custom software, macros, or scripts may be used to gain initial access, access credentials, establish command-and-control communications, or render the adversary’s desired impact, such as data encryption for ransomware. MalwareScore is a prevention in Elastic Endpoint Security that detects these malicious files via a trained machine learning model.
</p>
<p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt3cb8a6fb972d45e9/5ddc4df66afb810c2d465f63/ml-cybersecurity-ml-model-blog.png" data-sys-asset-uid="blt3cb8a6fb972d45e9" alt="MalwareScore machine learning model" style="display: block; margin: auto;">
</p><figcaption>Parsing files effectively to generate features and train MalwareScore models
</figcaption>
<p>Writing rules or heuristics for every type of known malicious software tool is not sustainable, especially as malware variants evolve. Instead, MalwareScore leverages the scale and automation benefits of machine learning. MalwareScore models are trained to generalize from patterns discovered in large corpora of malicious and benign files: exceeding 150M files for Windows executable files for recent model versions.
</p>
<p>MalwareScore consists of three malicious software models: Windows PE files, macro-based Office documents, and macOS binaries (Mach-O). We apply domain expertise from our threat intelligence team to identify features of these files important in determining whether the file is likely good or bad. For instance, Windows PE files are summarized by features that are derived from aspects of the file that include:
</p>
<ul>
	<li>Header and rich signature</li>
	<li>File section characteristics</li>
	<li>Version data</li>
	<li>Debug info included by the compiler</li>
	<li>Imported and exported functions</li>
	<li>The overlay (appended to the file)</li>
	<li>Resources section</li>
	<li>The entry point</li>
	<li>Thread local storage callbacks</li>
	<li>The distribution of byte patterns in the file</li>
</ul>
<p>Mach-O binaries are summarized from similar information. Macros embedded within Office documents are characterized by both macro content and document properties.
</p>
<p>Features like those listed above are readily converted into a format (vector of numbers) that can be ingested and analyzed by machine learning algorithms. MalwareScore leverages <a href="https://en.wikipedia.org/wiki/Gradient_boosting">gradient boosting decision trees</a> because these model types offer a nice combination of excellent efficacy (high detection rate with a low false positive rate) and computational simplicity (essentially, a series of if/then statements). Indeed, the models are designed to be lightweight (roughly 5MB) with low CPU utilization.
</p>
<p>MalwareScore is a supervised machine learning model, meaning that each file is paired with a malicious or benign label. The model is trained to minimize prediction errors of these target labels on the training set, with model regularization employed so that the model <em>generalizes</em> to samples not in the training set, rather than just <em>memorizing</em>. While the features and the model type are important, the richness and distribution of the training data and the quality of the labels are fundamental to providing a good model.
</p>
<p><strong></strong>
</p>
<h2><strong>What’s next?</strong></h2>
<p>MalwareScore-supervised machine learning models bring an important element of protection to the Elastic Stack. Users who install Elastic Endpoint Security are protected against malware as part of a layered protection strategy that includes behavioral rules. You can try a recent version of MalwareScore in <a href="https://www.virustotal.com/gui/home/search">VirusTotal</a>. Upload or search for a PE, macro, or Mach-O file, and view the results yourself!
</p>An even broader security and operational perspective can be attained in Elastic SIEM. In addition to endpoint events collected by Elastic Endpoint Security or other technologies, the SIEM app may include audit events, authentication logs, and firewall logs in holistic detail. In the next blog, we’ll discuss how machine learning in Elastic SIEM can be used to expose unknown threats via anomaly detection in an <em>unsupervised learning</em> paradigm. This allows you to protect against behaviors at the endpoint, and discover unexpected behavior in Elastic SIEM.

ml-cybersecurity-blog-banner.jpg

ml-cybersecurity-blog-thumb.png

Machine learning for cybersecurity: only as effective as your implementation

Before making a major purchase, chances are you shop around, compare products with a critical eye, and rely heavily on the experiences and opinions of people

<p>Before making a major purchase, chances are you shop around, compare products with a critical eye, and rely heavily on the experiences and opinions of people you trust to inform your buying decision. For example, before purchasing a car, customers often turn to friends and third-party reviews, such as <a href="https://www.consumerreports.org/cro/index.htm">Consumer Reports</a>, <a href="https://www.kbb.com/">Kelley Blue Book</a>, or the wisdom of crowds captured in sites such as <a href="https://www.yelp.com/">Yelp</a> and <a href="http://www.angieslist.com/">Angie’s List</a>. These sources can validate claims made by the automobile manufacturer or salesman, and are essential to help inform consumers and provide transparency.
</p><p>But what happens if we’re talking about computer security instead of cars? A consumer seeking a third-party review might feel somewhat bewildered for a few key reasons. First, some vendors may not participate in public third-party testing. In addition, the methodologies employed by testing companies may not be fully transparent to customers or even agreed upon by those vendors whose products are being evaluated. More transparency is required to incentivize both vendors and testers to adopt uniform and mutually understood standards for third-party evaluations, as occurs in many other industries. The expanded participation in third-party assessments by vendors, and an improvement of third-party tests, can best be achieved by fully adopting testing standards, such as those drafted by the Anti-Malware Testing Standards Organization (AMTSO), of which <a href="https://www.endgame.com/news/press-releases/endgame-joins-anti-malware-testing-standards-organization-amtso">Endgame is a participating member</a>. These assessments are a key step toward providing better transparency across the industry, and arming consumers with the required knowledge to make more informed decisions.
</p> <h2>Moving Beyond Self-Evaluations</h2><p>At Endgame, we maniacally and rigorously test our platform internally in environments that we have set up to mimic conditions experienced by our customers, including nascent threats. We exercise our detection engines, probe our protection mechanisms, even proactively and adversarially <a href="https://www.blackhat.com/us-17/briefings.html#bot-vs.-bot-for-evading-machine-learning-malware-detection">probe</a> our machine learning models for blind spots. As such, we believe our product provides superior protection to our customers. However, moving beyond self-evaluations is critical for at least three reasons.
</p><p>First, we’re not naïve to the fact that---subtly, but importantly---our product has been built to protect against threats that fit our understanding of the contemporary threat landscape. We have designed our product to generalize to evolving threats using behavioral-based and signatureless detections. Layered protections minimize the impact of a miss at any single point in the attack chain. As such, we’re confident in our ability to protect customers. It’s probably safe to assume that most mature vendors believe that their product is superior to competitors based on self-critical internal evaluations and subsequent improvements. We certainly do. But dataset bias is real, and by nature invisible. So, it behooves the rigorous and objectively-minded to submit to third-party testing, even when the testing results are never made public. It explicitly benefits the vendor by identifying any hidden weaknesses and strengths.
</p><p>Related is that fact that internal testing methodologies create a conflict of interest with inherent bias, either innocently or purposefully, because the vendor benefits from self-assigned high grades. Vendors optimize to their success metrics. Methodological massaging to boost metrics is a direct consequence of financial incentives to broadcast the metrics’ supremacy.  Of course, anyone can score well on a test they write for themselves. As amateur statistician Homer Simpson rightly <a href="https://youtu.be/7tzfl1wTemM">noted</a>, “people can come up with statistics to prove anything.” This notion of <a href="https://www.theatlantic.com/entertainment/archive/2012/05/10-years-after-its-premiere-the-wire-feels-dated-and-thats-a-good-thing/257910/">juking the stats</a> was popularized on <a href="http://www.hbo.com/the-wire"><em>The Wire</em></a>, but reflects a serious challenge when defining testing methodology and evaluation criteria. Thus, integrity compels an honest vendor to participate in independent third-party tests.
</p><p>Thirdly, publishing test results is critical to the consumer. While many customers have the resources to conduct their own extremely rigorous product evaluation, many must rely on third-party tests as an objective confirmation that products protect as advertised. For vendors who do not participate in public tests, the consumer is left to wonder whether private test results (if any) were unsatisfactory or would otherwise be embarrassing to the vendor if published.
</p><p>To summarize, vendors should participate in public third-party tests with motivations of objectivity, integrity, credibility, in addition to the obvious value-add as a critical comparative tool for customers. Just like the attackers, these tests also must evolve to evaluate against the broader range of attacker techniques in addition to malware, such as fileless attacks. Customers can and should demand this. Endgame is proud to participate in public third-party testing, including <a href="https://www.endgame.com/news/press-releases/independent-testing-confirms-endgame-uniquely-positioned-replace-av-converged">AV Comparatives</a> and <a href="https://www.endgame.com/news/press-releases/endgame-machine-learning-engine-achieves-100-independent-malware-certification">SE Labs</a>, and we look forward to sharing results of ongoing third-party public testing in the near future.
</p><h2>But Who is Testing the Test?</h2><p>Just as imperative as the need for vendors to participate in independent third-party testing, the security community must also ensure credibility in how tests are performed. From a customer's perspective, there are several elements of testing in infosec, especially related to potential conflicts of interest, that should at least raise an eyebrow. To paraphrase Dennis Batchelder, current president of AMTSO, at the most recent AMTSO meeting, in what other industry do you find the following scenarios?
</p><ul>
	<li>A testing organization that is evaluating a product may get paid by the vendor (or by a competitor) that is authorizing the test.</li>
	<li>A testing organization may test an unwilling vendor’s product for a use case or in an environment not intended by the vendor.</li>
	<li>A testing organization may receive significant input (e.g., malware samples) from the vendor being tested.</li>
	<li>A vendor may monitor or influence the outcome of the test while it is being performed, even changing the behavior of the product while it is being evaluated.</li>
	<li>A vendor may pay the testing organization for additional privileges or upgrades to the basic test.</li>
</ul><p>Let’s be clear, here. Nonprofit testing is important, but it is not the only option. Testing organizations are expending significant effort to develop realistic testing methodologies, which requires resources. At Endgame, we pay testers to test our product. Furthermore, the methodology by some testers may not be perfectly aligned with Endgame’s protection strategy, and we appreciate testers who seek our input into how to exercise our product’s functionality. Customers must be able to trust that money isn’t driving the outcomes of the test, and that the relationship of the vendor to tester is indeed independent rather than influential. In both cases---payment and methodology refinement---the best way to ensure that these relationships don’t undermine customers’ interests is for both parties to agree to the highest standards of transparency.
</p><p>Specifically for this reason, in May AMTSO adopted a draft standards document that outlines appropriate protocol for testers and vendors when evaluating anti-malware solutions. Like many technical standards documents (e.g., <a href="https://www.iso.org/isoiec-27001-information-security.html">ISO 27001</a>), the standards do not mandate specific implementation details. Rather, the document establishes a foundational protocol that promotes tests that are impartial, transparent, and notably address the inherent conflicts of interest present in our industry. Without this basic foundation that outlines protocols for fairness and transparency, more detailed standards aimed to improve testing quality are a proverbial house upon the sand. When testing methodology is transparent and becomes the subject of public scrutiny, testing quality is bound to follow.
</p><p>The draft status of the standards is intended to help AMTSO, testers, and vendors work through the implementation details in providing transparency and rules of engagement for vendors and testers. Through this process, AMTSO will revise the draft as needed. Once adopted, AMTSO (comprised of a consortium of vendors and testers and interested parties) then becomes an organization that, indeed, can implement testing standards, which have notably been absent in its decade-long history.
</p><p>Endgame is proud to support AMTSO in the organization’s effort to set standards of transparency, fairness and impartiality in third-party validation of security products. I am hopeful that after any necessary minor revisions revealed during the exercise of the draft, AMTSO member organizations will come to a consensus and fully adopt the testing protocol standards document. Similarly, Endgame is committed to additional tests that extend beyond malware to cover a broad range of attacker techniques. Customers deserve it.
</p><h2>Moving Forward with Openness</h2><p>The call to vendors and testers for openness and transparency is an invitation to somewhat unfamiliar territory in infosec. As Nate Fick, our CEO, sharply <a href="https://www.endgame.com/blog/executive-blog/whats-next-cybersecurity-community-video">called out</a> in a 2017 New America conference keynote: “Security is bedeviled by a dark arts culture that’s both self-serving and wrong. Security is no more a dark art than finance or real estate or tax policy or animal husbandry...and to wrap itself in a cape of black magic is nothing more than self-importance and a vain attempt at job security. It’s bad for customers.”
</p><p>We are in a transformative stage of the infosec industry. As enterprises iterate on compliance with frameworks such as NIST, there is a parallel demand to ensure greater transparency of the products to ensure they perform as advertised. Just as Kelley Blue Book and Angie’s List have provided independent means to assess various products, so too is infosec moving in this direction. Driven by the expectation of methodologically rigorous and fair independent evaluation, vendors are participating in these assessments, and consumers are demanding them. While testing methodologies may not be perfect, openly participating in them is crucial, and that’s why we have been and will continue to be actively engaged in a number of public tests with various vendors. Together as members of AMTSO, we’ll adopt what are now draft standards, and can finally move the community forward, removing the ‘dark arts’ mystique of the industry, and provide protection assurances to consumers, corporations and national security assets. Transparency is key. We should expect nothing less.
</p>

blog-banner-generic-blue.png

blog-thumb-generic-blue.png

Transparency In Third-Party Testing

<p>As <a href="https://www.globenewswire.com/news-release/2019/08/09/1900054/0/en/Endgame-MRG-Effitas-and-VMRay-Partner-on-Machine-Learning-Static-Evasion-Contest.html">announced</a> at DEFCON’s AIVillage, Endgame is co-sponsoring (with <a href="https://www.mrg-effitas.com/">MRG-Effitas</a> and <a href="https://www.vmray.com/">VM-Ray</a>) the <a href="https://evademalwareml.io/">Machine Learning Static Evasion Competition</a>. Contestants construct a white-box <a href="https://en.wikipedia.org/wiki/Adversarial_machine_learning#Evasion_attacks">evasion attack</a> with access to <a href="https://github.com/EndgameInc/malware_evasion_competition">open source code</a> for model inference and model parameters. The challenge: modify fifty malicious binaries to evade up to three open source malware models. The catch: the modified malware samples must retain their original functionality. The prize: the contestant/team to produce the most evasions and publish the winning solution will win an <a href="https://www.nvidia.com/en-us/titan/titan-rtx/">NVIDIA Titan-RTX</a>, a powerful and popular GPU for training deep learning models.
</p><h3>Why this competition?
</h3><p>Some may question why Endgame is sponsoring a competition that overtly encourages participants to evade endpoint security protections. After all, Endgame’s <a href="https://www.endgame.com/blog/technical-blog/world-meet-malwarescore">MalwareScore</a>™ is itself a static pre-execution antimalware engine built on machine learning (ML). At Endgame, we have long <a href="https://www.endgame.com/blog/technical-blog/why-we-release-our-research">espoused the view</a> that there is no security by obscurity, and that self- and public testing are <a href="https://www.endgame.com/blog/technical-blog/transparency-third-party-testing">more than hygienic</a>. Discovering and publicly sharing evasion strategies that capable contestants discover is good for security.
</p><p>The competition is unrelated to the <a href="https://skylightcyber.com/2019/07/18/cylance-i-kill-you/">recent evasion</a> by security researchers of commercial ML endpoint protection software. Although CylancePROTECT® was the target of the recent bypass, one should keep in mind that with enough work, any one protection component–ML or not–can be blatantly bypassed or carefully sidestepped. This is why security products should adhere to a layered protection strategy: should an attacker sidestep one defense, there are a host of other traps set with a hair trigger.
</p><p>In reality, the foundation for this competition began <a href="https://www.securityweek.com/bot-vs-bot-never-ending-cycle-improving-artificial-intelligence">many years ago</a> with Endgame adversarial machine learning research to create <a href="http://www.devtalk.tv/def-con-25-hyrum-anderson-evading-next-gen-av-using-ai_473c334e2.html">carefully-crafted malware perturbations</a> that evade machine learning models.
</p><p>In academic circles, adversarial machine learning has largely been confined to computer vision models, wherein image pixels are subtly modified to preserve human recognition, but exploit worst-case conditions of the model to achieve <a href="https://mashable.com/2017/11/02/mit-researchers-fool-google-ai-program/">catastrophic miscategorizations</a>. With each new attack strategy also comes proposals for making machine learning more robust against these attacks.
</p><p>But malware, as a structured input, is different than images. It’s harder. It’s mathematically inconvenient. And as such, we want to draw interest to this unique adversarial model. Simply put, even though Portable Executables (PE) files are a sequence of bytes in the same way an image is a sequence of pixels, they differ starkly in that when you slightly change an image pixel, the “imagey-ness” is preserved, but when you modify a byte in a PE file, you may very well corrupt the file format or destroy the program’s functionality. So, attackers must carefully constrain how they modify the PE byte stream. This point deserves to be highlighted to the academic community.
</p><p>Ultimately, defenders benefit by understanding the space of functionality-preserving mutations that an attacker might apply. In reality, attacker evasion techniques more often include source code modifications or compile-time tweaks, but the model we present represents a useful framework for reasoning about PE modifications in the same framework that academics reason about pixel perturbations. The hope is that by exploring the space of adversarial perturbations, defender models can have the benefit of having anticipated some part of the evasive repertoire of an adversary, and build robustness.
</p><h3>A useful and fun game
</h3><p>Contestants will attempt to evade three open source models. <a href="https://arxiv.org/pdf/1710.09435.pdf">MalConv</a> is an end-to-end deep learning model. The <a href="https://arxiv.org/pdf/1806.06108.pdf">non-negative MalConv</a> model has an identical structure to MalConv, but is constrained to have non-negative weights which forces the model to only look for evidence of maliciousness rather than evidence of both malicious and benign byte sequences.  Both of these models are end-to-end deep learning models, upon which most of the recent adversarial machine learning literature has been focused. The <a href="https://www.endgame.com/blog/technical-blog/extending-ember">recently updated</a> EMBER <a href="https://lightgbm.readthedocs.io/en/latest/">LightGBM</a> model is the third model, and operates on features extracted from an input binary. Each of the models was trained on the <a href="https://github.com/EndgameInc/ember">EMBER 2018 hashes</a>.
</p><p>It’s important to note that none of the models, as presented, are production-worthy models. In particular, these are “naked” models (e.g., no whitelist/blacklist) trained on comparatively small datasets, and the thresholds have been adjusted to detect each of the competition malware samples.  Thus, the EMBER model has a false positive rate of less than 5:1000, while the deep learning models have false positives rates exceeding 1:2. Nevertheless, the models represent useful targets for exploring and understanding successful evasive techniques.
</p><h3>A final word
</h3><p>Even with the chance of evasion, Machine Learning models ostensibly offer excellent detection rates at low false positive rates for detecting malware before it executes. Importantly, machine learning generalizes to new samples, evolving families and polymorphic strains. Machine Learning is much less brittle than signature-based approaches that memorize known threats, but miss subtle perturbations to those same samples. For this reason, machine learning has been an overwhelmingly positive development for information security.
</p><p>We thank our partners for the tremendous amount of work and resources they’ve contributed to make this a viable competition. Watch this space for competition results.
</p>

Machine Learning Static Evasion Competition

Today, we’d like to announce a new release of EMBER (Endgame Malware BEnchmark for Research), our open source benchmark dataset.

<p>Last year, Endgame released an open source benchmark dataset called EMBER (<a href="https://github.com/endgameinc/ember">Endgame Malware BEnchmark for Research</a>). EMBER contains 1.1 million portable executable (PE file) sha256 hashes scanned in or before 2017, features extracted from those PE files, a benchmark model, and a code repository that makes it easy to work with this data. Since then, researchers have been able to use this dataset to quantify how quickly models degrade [1], investigate how labels evolve over time [2], and even to investigate how malware classifiers are vulnerable to attack [3]. We were very pleased to see this response from the community, but were also aware of a couple areas where we thought EMBER could improve.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt7de14ba1b9f5af72/5dfd09ba7651f865a36d948c/endgame-ember-blog.png" style="background-color: initial; display: block; margin: auto;" data-sys-asset-uid="blt7de14ba1b9f5af72" alt="endgame-ember-blog.png"></p><p>Today, we’d like to announce a new release of EMBER that we built with those improvements in mind. This release is accompanied by a new set of 1 million PE files scanned in or before 2018 and an expanded feature set. Here are some of the differences between this release and the original EMBER:
</p><h2>More Difficult Dataset</h2><p>We found that the test set in the original EMBER 2017 dataset release was very easy to classify. The benchmark model on this data was trained with default LightGBM parameters and still achieved a 0.999 area under a receiver operating characteristic (ROC) curve. So this time around, we selected PE files to include while keeping in mind that we wanted to make the classification task harder. The benchmark model trained on the EMBER 2018 files is now optimized by a simple parameter grid search, but it still only achieves a 0.996 area under a ROC curve. This is still excellent performance, but we’re hoping this leaves more room for innovative and improved classification techniques to be developed.</p><h2>Expanded and Updated Feature Set</h2><p>This latest release of EMBER contains a new feature set that we are calling feature set version 2. The main difference between this feature set and version 1 features is that we are now using <a href="https://lief.quarkslab.com/">LIEF</a> version 0.9.0 instead of version 0.8.3. Feature calculations using the new version of LIEF are not guaranteed to be equivalent to those using the older version.
</p><p>Also, one of our goals for the features that we provided last year was that researchers could recreate decisions made by the <a href="https://sourceforge.net/adobe/malclassifier/wiki/Home/">Adobe Malware Classifier</a> without going back to the original PE file. It turns out that the original features were insufficient to reproduce the original work. In version 2 of our feature set, the new data directory features now allow that analysis.
</p><p>Thanks to a clustering analysis carried out on the original EMBER release, we found that the distribution of some feature values was very uneven. The main culprit were samples that had a large number of ordinal imports that were getting hashed to the exact same place. In order to smooth the feature space, ordinal imports are now hashed along with named imports in feature set version 2.
</p><p>After freezing the development of version 2 of our feature set, we calculated these new features for the original EMBER 2017 set of files. Those new features calculated on the old files are now publicly available for download in addition to the features from the new 2018 files.
</p><h2>PE File Selection</h2><p>Clustering analysis of EMBER 2017 also revealed extreme outliers and PE files that had exactly the same feature vector. Although this sort of dirty data is expected in the real world by any deployed static malware classifier, we wanted our benchmark dataset to better capture performance across a normalized view of the set of PE files. For this reason, we used this outlier and duplicate detection to clean some of the worst offenders before finalizing our selection for the 2018 feature set.
</p><p>Adding an additional year of labels and features onto the original data release opens the door to more longitudinal studies of model performance degradation and malware family evolution. While these possibilities are exciting, they are complicated by the fact that different logic was used in each year to select the set of files that were included in the dataset. The malware and benign samples are not sampled identically from a single distribution.&nbsp; Depending on the goal of the research, this difference may not matter. But researchers must be aware of this when forming and testing their hypotheses.
</p><p>It’s been inspiring to see all the interest in EMBER over the last year and a half. We’re hoping that this new release can further empower researchers to find new static classification techniques, quantify the performance of existing models, or simply to practice their data analysis skills. Please reach out if you have any questions or suggestions!
</p><p>[1] <a href="https://www.endgame.com/blog/technical-blog/beware-steep-decline-understanding-model-degradation-machine-learning-models">Understanding Model Degradation In Machine Learning Models</a>&nbsp;
</p><p>[2] <a href="https://www.camlis.org/david-krisiloff">Measure Twice, Quarantine Once: A Tale of Malware Labeling over Time</a>
</p><p>[3] <a href="https://www.camlis.org/bobby-filar">TreeHuggr: Discovering where tree-based classifiers are vulnerable to adversarial attack</a>
</p>

Extending EMBER

Openness and transparency are adjectives to which our community should aspire. But, where’s the bright line that one shouldn’t cross?

<p>Last week in an unprecedented move, researchers at <a href="https://openai.com/">OpenAI</a> stated that with the announcement of their powerful new language model, they <a href="https://blog.openai.com/better-language-models/#releasestrategy">would not be releasing</a> the dataset, code, or model weights due to safety and security concerns. The researchers cited founding principles in the OpenAI charter that predicts that “safety and security concerns will reduce our traditional publishing in the future.” The particular fear is that the GPT-2 model could be misused to generate at scale fake news articles, impersonate others, or for phishing campaigns. OpenAI did release a small GPT-2 model found <a href="https://github.com/openai/gpt-2">here</a>.
</p><p>Coincidentally, this news comes almost exactly one year since the release of the <a href="http://www.maliciousaireport.com/">Malicious Use of AI Report</a>, co-authored in part by researchers from OpenAI and Endgame. In some security circles such as AI Village, the discussion associated with the GPT-2 model non-release has <a href="https://aivillage.org/blog/2019/2/23/gpt-2-and-threatmodels">outweighed</a> that of the initial report. As we noted in our <a href="https://www.endgame.com/blog/technical-blog/malicious-use-artificial-intelligence-infosec">blog post</a> that coincided with the initial report, cybersecurity, with its <a href="https://www.endgame.com/blog/technical-blog/rsa-keynote-call-digital-norms">lack of norms</a>, lies at a critical inflection point in the potential misuse of machine learning in a broad array of fields that includes misinformation (for GPT-2).
</p><p>Endgame researchers, including authors of this post, have a history of releasing <a href="https://www.endgame.com/blog/technical-blog/new-open-source-repositories-data-scientists-infosec">models</a>, <a href="https://www.endgame.com/blog/technical-blog/introducing-ember-open-source-classifier-and-dataset">datasets</a>, <a href="https://github.com/endgameinc/gym-malware">code</a>, and <a href="https://arxiv.org/abs/1610.01969">papers</a> that could conceivably be used for malicious purposes. As co-authors with OpenAI on the Malicious Use of AI report, how do we justify these releases? What discussions and checks are in place when choosing to (or not to) release? Is a contraction in the sharing of infosec models and code to be expected?
</p> <strong><h3><strong>DECIDING TO RELEASE</strong></h3></strong><p>Each decision to release should come after a thoughtful debate about potential social and security impacts. In our case, none of our previously released research openly targeted flaws in software products, but rather highlights more generally potential blind spots or other causes of concern. We identified and discussed the costs attackers would need to consider to avoid releasing tools that exploited “low hanging fruit.” Importantly, for each red-team tactic, we also disclosed blue-team defenses.
</p><p>The series of releases has refined a few guiding principles about responsible release that are worth highlighting:
</p><ol>
	<li>Invite debate that deliberately includes parties with no vested interest in the code release or publication. Although they need not be fatalists, they can help you walk with fresh eyes through worst-case scenarios.</li>
	<li>Adopt long-standing and broadly recognized responsible disclosure guidelines that include early notification of impacted parties, generous time before disclosure, and generally acting in good faith.</li>
	<li>Expect and embrace public debate and pushback, which is an important socially-rooted check.</li>
	<li>Be willing to give the benefit of the doubt to inventors and authors on their decision to release or withhold. There may be additional details or discussion not made public that contribute to the decision. Even while maintaining the public dialog, don’t fall for the <a href="https://en.wikipedia.org/wiki/Fundamental_attribution_error">fundamental attribution error</a> in ascribing recklessness, greed or malintent.</li>
</ol> <strong><h3><strong>WHY DID WE RELEASE?</strong></h3></strong><p>The reality is most AI/ML research can be leveraged for benign or malicious uses. Technologies like <a href="https://en.wikipedia.org/wiki/Deepfake">DeepFakes</a> highlight that relatively innocuous research can pivot quickly into nightmare territory. For this reason it is important to acknowledge these concerns by committing to guiding principles such as those mentioned above. This is one net positive from OpenAI withholding the release of their GPT-2 model. Not without criticism, it set a precedent and brought the conversation into the public forum.
</p><p>Let’s take two recent examples from Endgame history where models, tools, and frameworks could have been leveraged by threat actors.
</p><p>The <a href="https://github.com/endgameinc/gym-malware">malware (evasion) gym</a> provided a reinforcement learning approach to manipulate malicious binaries to bypass antimalware machine learning models. This certainly constitutes red-team activity, and abiding by the principles above, we also <a href="https://arxiv.org/abs/1801.08917">disclosed</a> the best known defense for the attack. In this case, the machine learning model this project targeted was a “likeness” to commercial NGAV products, not a replica. Abiding by the spirit of responsible disclosure, it represented a very early warning proof-of-concept, not a gaping vulnerability. In reality, it would require a large effort to make this sort of attack successful for adversaries. The fact of the matter is that there are significantly easier, and less costly, ways to evade network or endpoint defenses.
</p><p><a href="https://github.com/endgameinc/ember">EMBER</a> is a benchmark dataset and anti-malware model that was released, in part, for adversarial research, and as such could be misused. Giving an adversary the “source code” to a modern machine learning antimalware architecture certainly can aid an attacker, but as in the other cases, education and transparency for researchers and defenders were ultimately paramount. We strongly believe that it is important to shine a bright light on the strengths and weaknesses of ML technologies in the information security domain. In fact, <a href="https://engineering.utulsa.edu/cyber-security-leadership-technology-summit/">showing weaknesses in models like EMBER can help Blue teams mitigate risks when facing a team</a>. This game theoretic approach is common across the infosec community and not one ML research should shy away from.
</p> <strong><h3><strong>A LESS LIBERAL FUTURE?</strong></h3></strong><p>Part of the machine learning community is emphasizing reproducibility with open code and model releases. It is all too easy to create a model that’s a little over trained and cherry picked, and the open release mitigates these issues. This is a wonderful aspect that has drawn many to the community, and this is the source of one of the biggest criticisms on OpenAI’s decision to not release the full model. Based on descriptions of the data set size and training cost, it’s less likely that GPT-2 is overtrained. But there is a legitimate call for open release.
</p><p>However, models that can produce images, text, and audio capable of fooling humans can be harmful. The common practice of reverse image searching a suspected bot’s profile image to find out if it’s a stock photo is completely defeated now with <a href="https://thispersondoesnotexist.com/">facial GANs</a>. There’s also the disturbing success that the red-team model <a href="https://www.blackhat.com/docs/us-16/materials/us-16-Seymour-Tully-Weaponizing-Data-Science-For-Social-Engineering-Automated-E2E-Spear-Phishing-On-Twitter.pdf">SNAP_R</a> had in phishing on twitter. These aren’t vulnerabilities in software that could be patched out, attacks that are based on these generative models are attacks on human perception. We can educate people to not follow links, and to not trust things they read on a Facebook post but it takes time and there will always be “unpatched” people who are new to the internet.
</p> <strong><h3><strong>FINAL THOUGHTS</strong></h3></strong><p>Healthy discussion about this continues, and we look forward to more formal forums where we can have discussions about releasing and mitigating dangerous models. Certainly, not every model need be readily available to advance the state of machine learning. If an author feels like their model could be dangerous, there should be a mechanism for them to submit it for review. If the review process validates the science and agrees that the model is dangerous, a report of the model needs to be published which includes a mitigation strategy. It is impossible to “put the genie back in the bottle” once a dangerous model is created. Nation state level actors have the resources to reproduce the full GPT-2, but we should probably not let every script kiddie on the internet have access to it.
</p><p>Security sits in an unenviable position. As we have <a href="https://www.endgame.com/blog/technical-blog/transparency-third-party-testing">argued before</a>, openness and transparency are adjectives to which our community should aspire. But, where’s the bright line that one shouldn’t cross? We think that line becomes crisper when one abides by the simple guiding principles as we’ve outlined here.
</p>

Why we release our security research

Presenting a powerful technique to detect domains created by domain generation algorithms (DGAs) that performs far better than “state-of-the-art” methods.

<p>The presence of domain names created by a Domain Generation Algorithm (DGA) is a telling indicator of compromise. For example, the domain <em>xeogrhxquuubt.com</em> is a DGA generated domain created by the Cryptolocker ransomware. If a process attempts to connect to this domain, then your network is probably infected with ransomware. Domain blacklists are commonly used in security to prevent connections to such domains, but this blacklisting approach doesn’t generalize to new strains of malware or modified DGA algorithms. We have been researching DGAs extensively at Endgame, and recently posted a <a href="https://arxiv.org/abs/1611.00791">paper on arxiv</a> that describes our ability to predict domain generation algorithms using deep learning.
</p><p>In this blogpost, we’ll briefly summarize our paper, presenting a basic yet powerful technique to detect DGA generated domains that performs far better than “state-of-the-art” techniques presented in academic conferences and journals. “How?” you may ask. Our approach is to use neural networks (more popularly called deep learning) and more specifically, Long Short-Term Memory networks (LSTMs). We’ll briefly discuss the benefits of this specific form of deep learning and then cover our simple yet powerful approach for detecting DGA algorithms.
</p><p>If you are unfamiliar with machine learning, you may want to jump over to our <a href="https://www.endgame.com/blog/machine-learning-you-gotta-tame-beast-you-let-it-out-its-cage">three part intro series</a> on machine learning before continuing on.
</p><h2>The Benefits of Long Short-Term Memory Networks</h2><p>Deep learning is a recent buzzword in the machine learning community. Deep learning refers to deeply-layered neural networks (one type of machine learning model), in which feature representations are learned by the model rather than hand-crafted by a user. With roots that go back decades, deep learning has become wildly popular in the last four-five years in large part due to improvements in hardware (such as improved parallel processing with GPUs) and optimization tricks that make training complex networks feasible. LSTMs are one such trick for implementing recurrent neural networks, meaning a neural network that contains cycles. LSTMs are extremely good at learning patterns in long sequences, such as text and speech. In our case, we use them to learn patterns in sequences of characters (domain names) so that we can classify them as DGA-generated or not DGA generated.
</p><p>The benefit of using deep learning over more conventional machine learning algorithms is that we do not require any feature engineering. Conventional approaches generate a long list of features (such as length, vowel to consonant ratio, and n-gram counts) and use these features to distinguish between DGA-generated and not DGA-generated domains (our colleague wrote about this in a previous <a href="https://www.endgame.com/blog/hunting-networks-part-2-higher-order-patterns">post</a>). If an adversary knows your features, they can easily update their DGA to avoid detection. This requires us, as security professionals, to go through a long and arduous process of creating new features to stay ahead. In deep learning, on the other hand, the model learns the feature representations automatically allowing our models to adapt more quickly to an ever-changing adversary.
</p><p>Another advantage to our technique is that we classify solely on the domain name without the use of any contextual features such as NXDomains and domain reputation. Contextual features can be expensive to generate and often require additional infrastructure (such as network sensors and third party reputations systems). Surprisingly, LSTMs without contextual information perform significantly better than current state of the art techniques that do use contextual information! For those who want to know more about LSTMs, tutorials are plentiful. Two that we recommend: <a href="https://colah.github.io/posts/2015-08-Understanding-LSTMs/">colah’s blog</a> and <a href="http://deeplearning.net/tutorial/lstm.html">deeplearning.net</a>, and of course our <a href="https://arxiv.org/abs/1611.00791">paper</a> on arxiv.
</p><h2>What is a DGA?</h2><p>First, a quick note on what a DGA is and why detecting DGAs is important. Adversaries often use domain names to connect malware to command and control servers. These domain names are coded into malware and give the adversary some flexibility in where the C2 is actually hosted in that it’s easy for them to update the domain name to point to a new IP. However, hardcoded domain names are easy to blacklist or sinkhole.
</p><p>Adversaries use DGAs to evade blacklisting and sinkholing by creating an algorithm (DGA) that creates pseudorandom strings that can be used as domain names. Pseudorandom means that the sequences of strings appear to be random but are actually repeatable given some initial state (or seed). This algorithm is used by both malware that runs on a victim’s machine as well as on some remote software used by the adversary.
</p><p>On the adversary side, the attacker runs the algorithm and randomly selects a small number of domains (sometimes just one) which he knows will be predicted by the DGA, registers them, and points them at C2 servers. On the victim side, malware runs the DGA and checks an outputted domain to see if it’s live. If a domain is registered, the malware uses this domain as its Command and Control (C2) server. If not, it checks another. This can happen hundreds or thousands of times.
</p><p>If security researchers gather samples and can reverse engineer the DGA, they can generate lists of all possible domains or pre-register domains which will be predicted in the future and use them for blacklisting or sinkholing, but this doesn’t scale particularly well since thousands or more domains can be generated by a DGA for a given day, with an entirely new list each following day. As you can see, a more generalized approach for predicting whether a given domain name is likely generated by a DGA would be ideal. That generalized approach is what we’re describing here.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltff6e7b941805f9d6/5e0e011df5916f4381bc9717/endgame-detecting-dgas-discovery-process-blog.jpeg" data-sys-asset-uid="bltff6e7b941805f9d6" alt="endgame-detecting-dgas-discovery-process-blog.jpeg" style="display: block; margin: auto;"></p><figcaption>Figure 1 demonstrates the discovery process used by many types of malware. In this case, the malware attempts three domains: asdfg.com, wedcf.com, and bjgkre.com. The first two domains are not registered and receive an NXDomain response from the DNS server. The third domain is registered and the malware uses this domain to call home.
</figcaption><h2>Building the LSTM</h2><h3>Training Data</h3><p>Any machine learning model requires training data. Luckily, training data for this task is easy to find. We used the Alexa top 1 million sites for <a href="https://s3.amazonaws.com/alexa-static/top-1m.csv.zip">benign data</a>. We also put together a few DGA algorithms in Python that you can grab on our <a href="https://github.com/endgameinc/dga_predict">github site</a>. We use these to generate malicious data. These will generate all the data needed to reproduce the results in this blog.
</p><h3>Tools and Frameworks</h3><p>We used the <a href="https://keras.io/">Keras</a> toolbox, which is a Python library that makes coding neural networks easy. There are other tools that we like, but Keras is the easiest to demo and understand. Keras is stable and production ready as it uses either <a href="http://deeplearning.net/software/theano/">Theano</a> or <a href="https://www.tensorflow.org/">TensorFlow</a> under the hood.
</p><h3>Model Code</h3><p>The following is our Python code for building our model:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blted5613c682037f91/5e0e015d4dfaba3e76b44caa/endgame-detecting-dgas-python-model-blog.png" data-sys-asset-uid="blted5613c682037f91" alt="endgame-detecting-dgas-python-model-blog.png" style="display: block; margin: auto;"></p><p>Yup, that’s it! Here is a very simple breakdown of what we’re doing (with links to additional information):
</p><ul>
	<li>We create a basic neural network model on the first line. The next line adds an embedding layer. This layer converts each character into a vector of 128 floats (128 is not a magical number. We chose it as it gave us the best numbers consistently). Each character essentially goes through a lookup once this layer is trained (input character and output 128 floats). <em>max_features</em> defines the number of valid characters. <em>input_length</em> is the maximum length string that we will ever pass to our neural network.</li>
	<li>The next line adds an LSTM layer. This is the main workhorse of our technique. <em>128</em> represents the number of dimensions in our internal state (this happens to be the same size as our previous embedding layer by coincidence). More dimensions mean a larger more descriptive model, and we found 128 to work quite well for our task at hand.</li>
	<li>The <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network#Dropout"><em>Dropout</em></a> layer is a trick used in deep learning to prevent overtraining. You can probably remove this, but we found it useful.</li>
	<li>This Dropout layer precedes a <em>Dense</em> layer (<a href="https://en.wikipedia.org/wiki/Convolutional_neural_network#Fully_connected_layer">fully connected layer</a>) of size 1.</li>
	<li>We added a <a href="https://en.wikipedia.org/wiki/Sigmoid_function">sigmoid activation</a> function to squash the output of this layer between 0 and 1, which represent, respectively, benign and malicious.</li>
	<li>We optimize using the <a href="https://en.wikipedia.org/wiki/Cross_entropy">cross entropy loss</a> function with the <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent#RMSProp">RMSProp</a> optimizer. RMSProp is a variant of <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent">stochastic gradient descent</a> and tends to work very well for recurrent neural networks.</li>
</ul><h2>Preprocessing Code</h2><p>Before we start training, we must do some basic data preprocessing. Each string should be converted into an array of ints that represents each possible character. This encoding is arbitrary, but should start at 1 (we reserve 0 for the end-of-sequence token) and be contiguous. An example of how this can be done is given below.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb3b552010d417f4f/5e0e01945f3f935f859311ff/endgame-detecting-dgas-converting-string-blog.png" data-sys-asset-uid="bltb3b552010d417f4f" alt="endgame-detecting-dgas-converting-string-blog.png" style="display: block; margin: auto;"></p><p>Next, we pad each array of ints to the same length. Padding is a requirement of our toolboxes to better optimize calculations (theoretically, no padding is needed for LSTMs). Fortunately, Keras has a nice function for this:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt945811a5c049fe8a/5e0e01b7631b8140b1f058bb/endgame-detecting-dgas-padding-blog.png" data-sys-asset-uid="blt945811a5c049fe8a" alt="endgame-detecting-dgas-padding-blog.png" style="display: block; margin: auto;"></p><p><em>maxlen</em> represents the length that each array should be. This function pads with 0’s and crops when an array is too long. It’s important that your integer encoding from earlier starts at 1 as the LSTM should learn the difference between padding and characters.
</p><p>From here, we can divide out our testing and training set, train, and evaluate our performance using a ROC curve.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt524ea7fd61ab74f7/5e0e01e3631b8140b1f058c1/endgame-detecting-dgas-roc-curve-blog.png" data-sys-asset-uid="blt524ea7fd61ab74f7" alt="endgame-detecting-dgas-roc-curve-blog.png" style="display: block; margin: auto;"></p><h4></h4><h2>Comparisons</h2><p>We compared our simple LSTM technique to three other techniques in our arxiv <a href="https://arxiv.org/abs/1611.00791">paper</a>. To keep things simple for this blog, we only compare the results with a single method that uses <a href="https://en.wikipedia.org/wiki/Bigram">bigram</a> distributions with <a href="https://en.wikipedia.org/wiki/Logistic_regression">logistic regression</a>. This technique also works better than the current state of the art (but not as good as an LSTM) and is surprisingly fast and simple. It is a more conventional feature-based approach where features are the histogram (or raw count) of all bigrams contained within a domain name. The code for this technique is also on our <a href="https://github.com/endgameinc/dga_predict">github</a> site.
</p><h2>Results</h2><p>We are finally ready to see some results! And here they are:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt776c174b97d89c32/5e0e020567e97c652d3c85c1/endgame-detecting-dgas-results-blog.png" data-sys-asset-uid="blt776c174b97d89c32" alt="endgame-detecting-dgas-results-blog.png" style="display: block; margin: auto;"></p><p>Nice! An AUC of 0.9977 with just a few lines of code! All of this is featureless with a simple and straightforward implementation. We actually did a much deeper analysis on a larger and more diverse dataset and observed 90% detection with a 1/10,000 false positive rate, and this can be combined with other approaches outside the scope of this post to improve detection even further. With our LSTM, we even did quite well on multiclass classification, meaning we could classify a piece of malware just by the domains it generated.
</p><h2>Conclusion</h2><p>We presented a simple technique using neural networks to detect DGAs. The technique uses no contextual information (such as NXDomains or domain reputation) and performs far better than state of the art. While this was a brief summary of our DGA research, feel free to read our complete <a href="https://arxiv.org/abs/1611.00791">paper on arxiv</a> and explore our code on <a href="https://github.com/endgameinc/dga_predict">github</a>. As always, comments and suggestions are welcome on our github site!
</p>

Using deep learning to detect DGAs

Machine learning is a fashionable buzzword right now in infosec, and is often referenced as the key to next-gen, signature-less security. But along with all of

<p>Machine learning is a fashionable buzzword right now in infosec, and is often referenced as the key to next-gen, signature-less security. But along with all of the hype and buzz, there also is a mind-blowing amount of misunderstanding surrounding machine learning in infosec. Machine learning isn't a silver bullet for all information security problems, and in fact can be detrimental if misinterpreted. For example, company X claims to block 99% of all malware, or company Y's intrusion detection will stop 99% of all attacks, yet customers see an overwhelming number of false positives. Where’s the disconnect? What do the accuracy numbers really mean? In fact, these simple statistics lose meaning without the proper context. However, many in the security community lack the basic fundamentals of machine learning, limiting the ability to separate the relevant and credible insights from the noise.
</p><p>To help bridge this gap, we're writing a series of machine learning-related blog posts to cut through the fluff and simplify the relevant fundamentals of machine learning for operationalization. In this first post, we provide a basic description of machine learning models. Infosec is ripe for a multitude of machine learning applications, but we’ll focus our overview on classifying malware, using this application to demonstrate how to compare models, train and test data, and how to interpret results. In subsequent blog posts, we'll compare the most prominent machine learning models for malware classification, and highlight a model framework that we believe works well for the malware-hunting paradigm on a lightweight endpoint sensor. While this post is focused on malware classification, the machine learning fundamentals presented are applicable across all domains of machine learning.
</p><h4>What is Machine Learning?</h4><p>In general, <a href="https://en.wikipedia.org/wiki/Machine_learning">machine learning</a> is about training a computer to make decisions. The computer learns to make decisions by observing patterns or structures in a dataset. In machine learning parlance, the output of training is called a model, which is an imperfect generalization of the dataset, and is used to make predictions on new data. Machine learning has many advantages, automating many aspects of data munging and analysis at scale. For example, executables are either benign or malicious, but it’s impossible to manually review all of them. A corporate system may contain millions of files that require classification and few, if any, companies have enough staff to manually inspect each file. Machine learning is perfect for this challenge. A machine learning model can classify millions of files in minutes and can generalize better than manually created rules and signatures.
</p><p><a href="https://en.wikipedia.org/wiki/Supervised_learning">Supervised learning</a> models are trained with examples to answer precisely these kinds of questions, such as, “is this file malicious?”. In this supervised learning setting, the training set may consist of two million Windows executables consisting of one million malware samples and one million benign samples. A machine will observe these samples and learn how to differentiate between benign and malicious files in order to answer the question. Typically, this decision is in the form of a score such as a single value between 0 and 1. The figure below demonstrates the creation and use of a supervised model.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt5c9e57f5994cb75b/5dfd522812af1140b8e770c2/endgame-anderson_many_samples.png" data-sys-asset-uid="blt5c9e57f5994cb75b" alt="endgame-anderson_many_samples.png" style="display: block; margin: auto;"></p><p>The model’s scores are converted to yes/no answers by way of a threshold. For example, if our scores ranged from 0 to 1, we may want to set a threshold of 0.5. Anything less than 0.5 is benign (“not malicious”), and everything equal or greater than 0.5 is malware (“is malicious”). However, models are rarely perfect (discussed later) and we may want to tweak this threshold for a more acceptable performance. For instance, perhaps missing malware is far worse than labeling a benign sample as malware. We might set the threshold lower, say 0.3. Or, maybe mislabeling in general is very bad, but our use case will allow for outputting unknown labels. In this case we can set two thresholds. We could choose to set everything below 0.3 as benign, everything above 0.7 as malicious, and everything else is unknown. A visualization of this is below. The important point here is that there is no one-size-fits-all solution, and it is essential to understand your data well and adjust models based on the use case and data constraints.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt7af9bcbaa833c395/5dfd5245631b8140b1f0418b/endgmae-_benign.png" data-sys-asset-uid="blt7af9bcbaa833c395" alt="endgmae-_benign.png" style="display: block; margin: auto;"></p><h4>How do we evaluate models?</h4><p>Metrics are necessary to compare models to determine which model might be most appropriate for a given application. The most obvious metric is accuracy, or the percentage of the decisions that your model gets right after you select appropriate thresholds on the model score. However, accuracy can be very misleading. For example, if 99.9% of your data is benign, then just blindly labeling everything as benign (no work required!) will achieve 99.9% accuracy. But obviously, that's not a useful model!
</p><p>Results should be reported in terms of false positive rate (FPR), true positive rate (TPR), and false negative rate (FNR). FPR measures the rate in which we label benign samples as malicious.  An FPR of 1/10 would mean that we classify 1 in 10 (or 10%) of all benign samples incorrectly as malicious. This number should be as close to 0 as possible. TPR measures the rate in which we label malicious samples as malicious. A TPR of 8/10 would mean that we classify 8 in 10 (or 80%) of all malicious samples as malicious. We want this number to be as close as possible to 1. FNR measures the rate in which we label malicious samples as benign and is the opposite of TPR (equal to 1-TPR). A 9/10 (9 in 10) TPR is the same as 1-9/10 or a 1/10 (1 in 10) FNR. Like FPR, we want this number to be as close to 0 as possible.
</p><p>Using these three measurements, our results don't look so great for the useless model above (always labeling a sample benign) evaluated only on benign samples. Our FPR rate is 0% (perfect!), but TPR is now 0%. Since we never label anything as positive, we will not get any false positives. But this also means we will never get a true positive, and while our accuracy may look great, our model is actually performing quite poorly.
</p><p>This tradeoff between FPR and TPR can be seen explicitly in a model's receiver operating characteristic (ROC) curve. A ROC curve for a given model shows the FPR / TPR tradeoff over all thresholds on the model’s score.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blta121ee4b82856003/5dfd526599f6f64041a2a0b1/endgame-false-positive.png" data-sys-asset-uid="blta121ee4b82856003" alt="endgame-false-positive.png" style="display: block; margin: auto;"></p><p>What FPR, TPR and FNR metrics really mean to a user greatly depends on scale. Let's say that we have a FPR of 0.001 (1/1000). Sounds great, right? Well, a Windows 7 x64 box has approximately 50,000 executables on a given system. Let’s say you have 40,000 endpoints across your network. Absent any actions to rectify the false positives, this model would produce approximately two million false positives if applied to all Windows executables on all systems across your entire network! If your workflow calls for even the most basic triage on alerts, which it should, this would be an inordinate amount of work.­­
</p><h4>Importance of Training and Testing Data</h4><p>So is the key to good machine learning a good metric? Nope! We also have to provide our learning stage with good training and testing data. Imagine all of our training malware samples update the registry, but none of the benign samples do. In this example, a model could generalize anything that touches the registry as malware, and anything that does not touch the registry is benign. This, of course, is wrong and our model will fail miserably in the real world. This example highlights a phenomena known as <a href="https://en.wikipedia.org/wiki/Overfitting">overtraining or overfitting</a> and occurs when our model becomes too specific to our training set and does not generalize. It also highlights a problem of bias, where our training/testing data over-represents a sub-population of the real-world data. For example, if the training data disproportionately is overpopulated with many samples from a small number of malware families, you may end up with a model that does a great job detecting new variants of those specific families but a lousy job detecting anything else.
</p><p>Why is this important? Overfitting and bias can lead to misleading performance.  Let's assume you are training your malware classification model on 5 families (benign + 4 different malware families). Let's also assume that you computed FPR, TPR, and FNR on a different set of samples consisting of the same 5 families. We'll also assume that your model is awesome and gets near perfect performance. Sounds great, right? Not so fast! What do you think will happen when you run this model in the real world and it’s forced to classify families of malware that it has never seen? If your answer is "fail miserably" then you're probably correct.
</p><p>Good performance metrics are not enough. The training/testing data must represent and function in the real world if you expect similar runtime performance as the performance measured during test time.
</p><h4>Acquiring Data</h4><p>So where do you get good train and test data? At Endgame we leverage a variety of sources ranging from public aggregators and corporate partnerships to our own globally distributed network of sensors. Not only do these sources provide a vast of amount of malicious samples, but they also aid in acquiring benign samples helping Endgame achieve a more diverse set of data. All of this allows us to gather real world data to best train our classifiers. However, your training data can never be perfect. In the case of malware, samples are always changing (such as new obfuscation techniques) and new families of malware will be deployed. To counteract this ever-evolving field, we must be diligent in collecting new data and retraining our models as often as possible. Without a constantly evolving model, we will have blinds spots and adversaries are sure to exploit them. Yes, the game of cat and mouse still exists when we use machine learning!
</p><h4>Acquiring Labels</h4><p>Now that your training/testing data is in the computer, we need to decide which samples are benign and which samples are malicious. Unlabeled or poorly labeled data will certainly lead to unsuccessful models in the real world. Malware classification raises some interesting labeling problems ranging from the adversary use of benign tools for nefarious means to new, ever-evolving malware that may go undetected.
</p><ul>
	<li>Unlabeled data. Data acquired from a sensor is often unlabeled, and presents a serious problem for machine learning. Unlabeled binaries require human expertise to reverse engineer and manually label the data, which is an expensive operation that does not scale well to the size of the training set needed for model creation.</li>
	<li>Incorrectly labeled data. Unfortunately, when classifying malware, the malware is often incorrectly labeled. Whether labeled by humans or machines, the necessary signatures may not be accessible or known, and therefore by default the malware is classified as benign. This will ultimately confuse the classifier and degrade performance in terms in FPR, TPR, and FNR.</li>
	<li>Nebulous or inconsistent definitions. There is no consistent or concrete definition of what constitutes malware. For example, it is common for attackers to use administrative tools to navigate a system post-attack. These tools aren't inherently malicious, but they often have the same capabilities as their malicious counterparts (e.g., the ability to send network commands, view processes, dump passwords) and are often used in attacks. On the flip side, they are readily available and often used on a system administrator's machine. Constantly calling them malicious may annoy those administrators enough to entirely ignore your model's output. Decisions have to be made on what labels, and how much weight, to give these sorts of programs during training to ensure models generalize instead of overfit, and support business operations instead of becoming a nuisance.</li>
</ul><p>The table below summarizes some primary challenges of labeling and their solutions.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt9650397c4dac997d/5dfd528467e97c652d3c6e97/endgame-major-challenges.jpeg" data-sys-asset-uid="blt9650397c4dac997d" alt="endgame-major-challenges.jpeg" style="display: block; margin: auto;"></p><h4>Conclusion
</h4><p>So remember, machine learning is awesome, but it is not a silver bullet. While machine learning isn’t a silver bullet and signatures and IOCs are not going away, some of the most relevant, impactful advances in infosec will stem from machine learning. Machine learning will help leapfrog our detection and prevention capabilities so they can better compete with modern attacker techniques, but it requires a lot of effort to build something useful, and, like all approaches, will never be perfect. Each iterative improvement to a machine learning model will be met with a novel technique by an adversary, which is why the models must evolve and adapt just as adversaries do.
</p><p>Evaluating the performance of machine learning models is complicated and results are often not as really, really, ridiculously good looking as the latest marketecture would have you believe. When it comes to security you should always be a skeptic and the same goes for machine learning. If you are not a domain expert just keep the following questions in mind the next time you are navigating the jungle of separating machine learning fact from fiction.
</p><p>We believe that machine learning has the potential to revolutionize the field of security. However, it does have its limitations. Knowing these limitations will allow you to deploy the best solutions, which will require both machine learning and conventional techniques such as signatures, rules, and technique-based detections. Now that you are armed with a high-level introduction to interpreting machine learning models, the next step is choosing the appropriate model. There are a myriad of tradeoffs to consider when selecting a model for a particular task and this will be the focus of our next post. Until then, keep this checklist handy and you too will be able to detect data science fact from the data science fiction.
</p>

Articles by Hyrum Anderson

Machine learning for cybersecurity: only as effective as your implementation

Transparency In Third-Party Testing

Machine Learning Static Evasion Competition

Extending EMBER

Why we release our security research

Using deep learning to detect DGAs

Machine Learning: You Gotta Tame The Beast Before You Let It Out Of Its Cage

It's A Bake-Off!: Navigating The Evolving World Of Machine Learning Models

Produits

Entreprise

Ressources