We’d like to announce the first release candidate of the Elastic Stack 8.0.0 and the continuation of the Pioneer Program. Give it a try and please share all the bugs you find on GitHub.

genericrelease.png

Elastic Stack 8.0.0-rc1 released

audit-letters-full-bleed.jpg

audit-clipboard-cc.jpg

Introducing Auditbeat: Ship Linux Audit Logs to Elasticsearch and More

container-loading-small.jpg

container-ship-rena-small.jpg

Monitoring Container Resource Usage with Metricbeat

How to use the Winlogbeat and Kibana to visualize logon events from Windows event logs.

winlogbeat-rdp-map-full-bleed.jpg

winlogbeat-account-usage-dashboard.jpg

Monitoring Windows Logons with Winlogbeat

<html><head></head><body><p><strong>Updated January 12, 2017:</strong> This post was updated to reflect changes in Packetbeat 5.x and Elasticsearch 5.x.
</p>


<p>Data observed from monitoring DNS traffic on a network can be used as an
indicator of compromise (IOC). This blog post will discuss how Elasticsearch and
Watcher can be used with Packetbeat to alert when possible malware activity is
detected.
</p>
<p><a href="/products/beats/packetbeat"><em>Packetbeat</em></a> is our open
source packet analyzer. It monitors the traffic on your network and indexes the
DNS requests and responses into Elasticsearch where aggregations can be used to
help make sense of the data.
</p>
<p><a href="/products/x-pack/alerting"><em>Alerting</em></a> (formerly <em>Watcher</em>)
is part of 
	<a href="/products/x-pack">X-Pack</a> and it provides alerting and
notifications based on changes in your data.
</p>
<p>There are many use cases for alerting on data collected by Packetbeat such as
alerting when the response times for web requests are above a threshold or when
there is spike in HTTP errors returned by your web servers. The alerting
described in this article has applications in network security. We are going to
look at one specific use case -- detecting data exfiltration over DNS tunnels.
</p>
<h2>Detecting DNS Tunnels</h2>
<p>Tunnels can be established over the DNS protocol to covertly move data or
provide a command and control channel for malware. Often this technique is used
to bypass the protections of corporate firewalls and proxy servers. Tunneling
works by encoding data in DNS requests and responses. The client issues a query
for a hostname and that query is eventually forwarded to the authoritative name
server associated with the domain.
</p>
<p><img src="https://api.contentstack.io/v2/assets/575e4d2d58208ba076e29b7d/download?uid=bltf505dadaa68339b8" data-sys-asset-uid="bltf505dadaa68339b8" alt="Packetbeat Deployment Architecture">
</p>
<p>There are a lot of different techniques that can be employed for detecting
such traffic. We are going to look at using the number of unique hostnames for a
domain as an IOC. DNS tunneling utilities must use a new hostname for each
request which leads to a much higher number of hostnames present for the
malicious domains in comparison to legitimate domains.
</p>
<h2>Packetbeat Setup</h2>
<p>The first step is to install Packetbeat and configure it to collect DNS
traffic. For this setup, the server running Packetbeat is connected to a port
mirror so that Packetbeat can observe all the traffic between the local network
and the Internet. The Packetbeat documentation has a great 
	<a href="/guide/en/beats/packetbeat/current/packetbeat-getting-started.html">Getting
Started guide
	</a> that explains installation and setup procedure, so I will just
show the configuration used.
</p>
<pre class="prettyprint"># /etc/packetbeat/packetbeat.yml
packetbeat.interfaces.device: en0
packetbeat.protocols.dns:
  ports: [53]
  include_authorities: true
  include_additionals: true
output.elasticsearch:
  hosts: [&quot;localhost:9200&quot;]
</pre>
<h2>Watch your DNS Traffic</h2>
<p>The complete
	<a href="https://github.com/elastic/examples/blob/master/Security%20Analytics/dns_tunnel_detection/unique_hostnames_watch.json">
	watch file</a> is stored in our
	<a href="https://github.com/elastic/examples/tree/master/Security%20Analytics/dns_tunnel_detection">elastic/examples</a>
	repository along with all of the supporting files shown here. We are going to
walk through the creation of this watch step-by-step.
</p>
<h2>Watch Trigger</h2>
<p>The watch trigger specifies when the execution should start. This watch is
scheduled to execute every 15 minutes.
</p>
<pre class="prettyprint">      &quot;trigger&quot;: {
         &quot;schedule&quot;: {
            &quot;interval&quot;: &quot;15m&quot;
         }
      },
</pre>
<h2>Watch Input</h2>
<p>This first step in creating this watch is to design a set of aggregations to
be used as the input to the watch. We want to find the cardinality of the
hostnames associated with each second-level domain
(e.g. 
	<code>badguy.co.</code>).
</p>
<p>We start with a query that has just two components, a time window and a
whitelist. The time window and whitelist can be customized. Find more on this in
the 
	<a href="#tuning">Tuning the Detector</a> section.
</p>
<p>Next we use a terms aggregation to create buckets for each second-level
domain. Then we apply a sub-aggregation to get the cardinality of the hostnames
within that bucket. Finally we apply a bucket selector aggregation to select
only the buckets having more than 200 unique hostnames. The watch will generate
an alert when the number of unique hostnames breaks this threshold.
</p>
<pre class="prettyprint">GET packetbeat-*/dns/_search
{
  &quot;query&quot;: {
    &quot;bool&quot;: {
      &quot;filter&quot;: {
        &quot;range&quot;: {
          &quot;@timestamp&quot;: {
            &quot;from&quot;: &quot;now-4h&quot;
          }
        }
      },
      &quot;must_not&quot;: {
        &quot;terms&quot;: {
          &quot;dns.question.etld_plus_one&quot;: [
            &quot;akadns.net.&quot;,
            &quot;amazonaws.com.&quot;,
            &quot;apple.com.&quot;,
            &quot;apple-dns.net.&quot;,
            &quot;cloudfront.net.&quot;,
            &quot;icloud.com.&quot;,
            &quot;in-addr.arpa.&quot;,
            &quot;google.com.&quot;,
            &quot;yahoo.com.&quot;
          ]
        }
      }
    }
  },
  &quot;size&quot;: 0,
  &quot;aggs&quot;: {
    &quot;by_domain&quot;: {
      &quot;terms&quot;: {
        &quot;size&quot;: 1000,
        &quot;field&quot;: &quot;dns.question.etld_plus_one&quot;
      },
      &quot;aggs&quot;: {
        &quot;unique_hostnames&quot;: {
          &quot;cardinality&quot;: {
            &quot;field&quot;: &quot;dns.question.name&quot;
          }
        },
        &quot;total_bytes_in&quot;: {
          &quot;sum&quot;: {
            &quot;field&quot;: &quot;bytes_in&quot;
          }
        },
        &quot;total_bytes_out&quot;: {
          &quot;sum&quot;: {
            &quot;field&quot;: &quot;bytes_out&quot;
          }
        },
        &quot;high_num_hostnames&quot;: {
          &quot;bucket_selector&quot;: {
            &quot;buckets_path&quot;: {
              &quot;unique_hostnames&quot;: &quot;unique_hostnames&quot;
            },
            &quot;script&quot;: &quot;params.unique_hostnames &gt; 200&quot;
          }
        }
      }
    }
  }
}
</pre>
<p>The query above relies on the <code>dns.question.etld_plus_one</code> field provided
by Packetbeat 5.x to bucket all requests for a single domain. Packetbeat
creates this field using an embedded copy of the 
	<a href="https://publicsuffix.org/">
	Public Suffix List</a>.
</p>
<h2>Watch Condition</h2>
<p>The watch condition is what determines if an alert is triggered. The
condition here is simple. This says that if any buckets were returned then
trigger the alert.
</p>
<pre class="prettyprint">      &quot;condition&quot;: {
         &quot;script&quot;: {
            &quot;inline&quot;: &quot;ctx.payload.aggregations.by_domain.buckets.size() &gt; 0&quot;
         }
      },
</pre>
<h2>Watch Actions</h2>
<p>The watch actions are executed after the condition is met, and define the
&quot;output&quot; of a watch. For this watch we are sending an email and also writing a
message to the Elasticsearch log. A&#xA0;
	<em>transform</em>&#xA0;script is being
used to manipulate the data so that it renders better in an email.
</p>
<pre class="prettyprint">        &quot;transform&quot;: {
           &quot;script&quot;: {
              &quot;file&quot;: &quot;dns_transform&quot;
           }
        },
        &quot;actions&quot;: {
           &quot;log_domains&quot;: {
              &quot;logging&quot;: {
                 &quot;text&quot;: &quot;The following domain(s) have a high number of unique hostnames: {{ctx.payload.alerts}}&quot;
              }
           },
           &quot;email_alert&quot; : {
            &quot;email&quot;: {
              &quot;to&quot;: &quot;&apos;John Doe &lt;john.doe@example.com&gt;&apos;&quot;,
              &quot;subject&quot;: &quot;Suspected DNS Tunnel Alert&quot;,
              &quot;body&quot;: &quot;The following domain(s) have a high number of unique hostnames: {{ctx.payload.alerts}}&quot;
            }
          }
        }
</pre>
<p>Below is the dns_transform Painless script. It should be placed into the
config/scripts directory of Elasticsearch.
</p>
<pre class="prettyprint">// File: config/scripts/dns_transform.painless
def alerts = ctx.payload.aggregations.by_domain.buckets.stream().collect(Collectors.toMap(p-&gt;p.key,item-&gt;[
        &quot;total_requests&quot; : item.doc_count,
        &quot;unique_hostnames&quot; : item.unique_hostnames.value,
        &quot;total_bytes_in&quot; : item.total_bytes_in.value,
        &quot;total_bytes_out&quot; : item.total_bytes_out.value,
        &quot;total_bytes&quot; : item.total_bytes_in.value + item.total_bytes_out.value
]));
return [&quot;alerts&quot;:alerts];
</pre>
<p>Here is a sample alert. Notice it contains a some additional metrics that can
be used to gauge the severity of the situation.
</p>
<pre class="prettyprint">Date: Fri, 16 Feb 2016 11:00:01 -0500 (EST)
From: Watcher &lt;watcher@example.com&gt;
Message-Id: &lt;201602161600.u0SG01ks024814@example.com&gt;
To: John Doe &lt;john.doe@example.com&gt;
Subject: Suspected DNS Tunnel Alert
The following domain(s) have a high number of unique hostnames:
{badguy.co.={total_requests=222, unique_hostnames=222, total_bytes_in=16716.0,
total_bytes_out=35161.0, total_bytes=51877.0}}
</pre>
<p>When this alert is received, the recipient can take the domain and do a
search using the Discover application in Kibana to find the network clients
responsible for the tunnel.
</p>
<h2>Testing and Results</h2>
<p><img src="https://api.contentstack.io/v2/assets/575e4d2dd8edd48f7693791e/download?uid=blt8aad375573dc7783" data-sys-asset-uid="blt8aad375573dc7783" alt="Unique FQDNs per Second Level Domain">
</p>
<p>This chart shows the top ten domains with the highest number of unique
hostnames over a period of 4 hours. This data was collected from a network with
about 100 devices. During that time window I replayed a network capture
containing a tunnel created by&#xA0;
	<a href="http://code.kryo.se/iodine/">iodine</a>. The tunnel which is operating
under the fictitious domain&#xA0;
	<code>pirate.sea</code>&#xA0;was up for just 20
seconds, and yet it has the highest number of domains.
</p>
<p>If the whitelist from the watch is applied then the tunnel really stands out
among the other domains as seen below.
</p>
<p><img src="https://api.contentstack.io/v2/assets/575e4d2d12798d8e381ff0be/download?uid=bltf1e4c3e0ba99f734" data-sys-asset-uid="bltf1e4c3e0ba99f734" alt="Unique FQDNs per Second Level Domain with Whitelist">
</p>
<h2 id="tuning">Tuning the Detector</h2>
<p>There are two variables that can be tuned -- the time window and the unique
hostnames threshold. A smaller time window can be used with a smaller threshold
to make the watch more sensitive to short duration tunnels. In a shorter time
window, domains not being used for tunneling will generally accumulate fewer
unique hostnames.
</p>
<p>Tunnels that move data slowly can be detected using a larger time window. But
using a larger time window means that valid domains with a lot of unique
hostnames, such as CDNs, will cause false positives. So if you use a large time
window you will likely need to add domains to the query&apos;s whitelist.
</p>
<h2>Conclusion</h2>
<p>It was fun combining Packetbeat and Watcher to look for DNS tunnels. Remember
&quot;defense in depth&quot; if you implement a solution like this. It is important to
layer your defenses so that if one layer fails there is another one in place to
detect.
</p></body></html>

tunnel-in-gaza.jpg

brick-tunnel.jpg

Detecting DNS Tunnels with Packetbeat and Watcher

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

headshot-andrew-kroh-300x300.jpg

Software Engineer

Andrew Kroh

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

El contenido de esta página no está disponible en el idioma seleccionado. Elastic está trabajando para garantizar que el contenido esté disponible en varios idiomas. Gracias por tu paciencia mientras trabajamos.

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Communication Preferences - ONLY for /communication-preferences

<p>Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/elastic-cloud-account-terms" target="_self">Condiciones de servicio</a>, y que acepta recibir las <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">comunicaciones seleccionadas</a> en los detalles de contacto que proporcionó anteriormente. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Serverless GDPR Text

<p>Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/serverless-preview-terms" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.</p>

<p>By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.</p>

Cloud Trial GDPR Text

<p>Al enviar esta información, aceptas los <a href="/es/agreements/global/cloud-services-monthly" rel="nofollow">Términos de servicio de Elastic</a>. Los datos personales se procesarán de conformidad con la <a href="/es/legal/privacy-statement">declaración de privacidad de Elastic</a>.
</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

<p>Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/terms-of-use" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.</p>

Hosted by

All Solutions

Follow us on Youtube

Watch now (no PT)

Articles by

Share by Email

Filters

All (no PT translation)

Contact information

See when this webinar starts in my time zone

Ver cuándo comienza este webinar en mi zona horaria

Thank you for your interest!

Register to Attend

Small image for

Register to Watch

Continue reading

Featured webinar

What to explore next...

Reset all

Load more press releases

Sign In to Attend

Share

Upcoming webinar

Learn more

Search Integrations

Share on Reddit

Share on Twitter

Share on Facebook

Load more news

View more posts

On-demand webinar

Follow us on Facebook

Author

Press Release

You'll also receive an email with related content.

You'll also receive an email with related content

Además, te enviaremos contenidos relevantes

Print

Contact Info

Overview

Video for

Read less

Register now

Share this story

See All Posts

See all top stories

View next

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

¿No puedes asistir? Regístrate y te enviaremos la grabación. También recibirás un correo electrónico con contenido relacionado.

Follow us on LinkedIn

Highlights

Explore similar demos

Follow us on Twitter

Share on LinkedIn

Load More

See more insights

Speakers

Close

Thank you for registering. We will send you a confirmation email soon.

Gracias por registrarte. Te enviaremos un correo electrónico de confirmación pronto.

Author

Articles by Andrew Kroh

Software Engineer, Elastic

Elastic Stack 8.0.0-rc1 released

Introducing Auditbeat: Ship Linux Audit Logs to Elasticsearch and More

Monitoring Container Resource Usage with Metricbeat

Monitoring Windows Logons with Winlogbeat

Detecting DNS Tunnels with Packetbeat and Watcher