<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/">
    <channel>
        <title>Elastic Security Labs - Articles by Erik-Jan de Kruijf</title>
        <link>https://www.elastic.co/security-labs</link>
        <description>Trusted security news &amp; research from the team at Elastic.</description>
        <lastBuildDate>Thu, 07 May 2026 17:32:30 GMT</lastBuildDate>
        <docs>https://validator.w3.org/feed/docs/rss2.html</docs>
        <generator>https://github.com/jpmonette/feed</generator>
        <image>
            <title>Elastic Security Labs - Articles by Erik-Jan de Kruijf</title>
            <url>https://www.elastic.co/security-labs/assets/security-labs-thumbnail.png</url>
            <link>https://www.elastic.co/security-labs</link>
        </image>
        <copyright>© 2026. elasticsearch B.V. All Rights Reserved</copyright>
        <item>
            <title><![CDATA[Detecting Web Server Probing & Fuzzing in Traefik with Automated Cloudflare Response]]></title>
            <link>https://www.elastic.co/security-labs/detecting-web-server-probing-and-fuzzing</link>
            <guid>detecting-web-server-probing-and-fuzzing</guid>
            <pubDate>Fri, 08 May 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[This article shows how a customized Elastic Security ES|QL detection rule can identify web server probing and fuzzing activity in Traefik logs and automatically block the attacking IP via Cloudflare.]]></description>
            <content:encoded><![CDATA[<h2>Introduction</h2>
<p>Self-hosted services exposed through a reverse proxy inevitably attract automated scanners probing for misconfigurations, admin panels, and vulnerable endpoints. In this article, I show how to turn routine <a href="https://traefik.io/traefik">Traefik</a> access logs into an active defensive control using Elastic Security and Cloudflare.</p>
<p>I use an out-of-the-box <a href="https://www.elastic.co/docs/explore-analyze/discover/try-esql">ES|QL</a> detection rule to identify <a href="https://elastic.github.io/detection-rules-explorer/rules/8383a8d0-008b-47a5-94e5-496629dc3590">web server discovery and fuzzing behavior</a>. When suspicious probing patterns are detected, an automated workflow immediately blocks the offending source IP at the edge via the Cloudflare API. The best part about this setup is that it scales effortlessly. By building this response plumbing once for fuzzing detection, I can attach the exact same block action to any other Elastic rule such as those catching SQL injections or file inclusion attempts. This transforms a basic logging pipeline into a highly adaptable perimeter defense.</p>
<h2>Background and the threat landscape</h2>
<p>My homelab setup utilizes Proxmox VE for containers and VMs. I use a Traefik reverse proxy, secured with <a href="https://www.authelia.com/">Authelia</a> for authentication, to allow external access without a VPN. Cloudflare, with proxy enabled, manages DNS.</p>
<p>For those less familiar with this specific stack, Traefik acts as the network's front door. When a web request arrives via Cloudflare, Traefik dynamically routes the traffic to the correct internal container while managing SSL certificates to keep the connections encrypted. However, before any traffic actually reaches those backend applications, it gets intercepted by Authelia. By leveraging Traefik's forward authentication feature, Authelia enforces Single Sign-On and Multi-Factor Authentication across the board. This means automated scanners and attackers cannot even reach the login screens of my internal services without passing through that initial secure portal.</p>
<p><img src="https://www.elastic.co/security-labs/assets/images/detecting-web-server-probing-and-fuzzing/image1.png" alt="Network diagram" title="Network diagram" /></p>
<p>To maintain visibility and security, I ingest these Traefik access logs into Elastic using the official integration. During routine monitoring, I've observed numerous HTTP 404 response codes originating from the same source IP addresses in these logs.</p>
<p>This pattern suggests potential web server probing or fuzzing traffic targeting vulnerabilities in applications that are not actually in use on my network. Examples of these targeted paths include <code>/wp-includes/mani.</code>, <code>/wp-content/plugins/all-in-one-wp-security-and-firewall/templates.php</code>, <code>/archive.php</code>, and <code>/wp-admin/includes/header.php</code>.</p>
<h3>Design philosophy: why not Fail2Ban?</h3>
<p>A common question in the homelab community is why not simply use local tools like <a href="https://github.com/fail2ban/fail2ban">Fail2Ban</a> or <a href="https://www.crowdsec.net/">CrowdSec</a> directly on the Traefik server. While those are excellent tools, orchestrating the response through Elastic Security and pushing the block to Cloudflare provides two major advantages. Dropping malicious traffic at the Cloudflare edge saves local bandwidth and keeps scanners off the home network entirely. Plus, orchestrating the response through Elastic gives us a single pane of glass for all security monitoring.</p>
<h2>Detection strategy and implementation strategy</h2>
<p>To effectively identify malicious reconnaissance, our strategy relies on analyzing the frequency of HTTP response codes at the proxy level. Specifically, we are looking for a high volume of 404 (Not Found) errors generated by a single source IP within a short time window, a classic indicator of directory fuzzing or vulnerability scanning.</p>
<p>While Elastic Security provides robust, out-of-the-box detection rules for this exact scenario, these rules require properly normalized ECS (Elastic Common Schema) data to function correctly. Detecting and mitigating these scans therefore requires a coordinated flow. To get this working, we need to ingest the Traefik logs, patch in the missing <code>host.name</code> field using a custom pipeline, and point the detection rule at our data.</p>
<h3>Threshold logic and tuning</h3>
<p>Our detection strategy shifts away from simple string matching, relying instead on statistical thresholds. The rule specifically monitors for denied or non-existent resources represented by HTTP 403 and 404 response codes and aggregates this activity by the originating source IP.</p>
<p>This behavior is governed by the final <code>where</code> statement in the query. By default, an alert only triggers if a source IP produces more than 500 errors across 250 distinct URI paths during the polling window. This dual-layered threshold is designed to eliminate false positives, ensuring that a single broken asset doesn't trigger a block while still identifying automated scripts that cycle through directory wordlists.</p>
<p>In a smaller homelab or smaller teams environment, these defaults are often too permissive. Since legitimate external traffic has no reason to hit non-existent admin panels on my network, I adjusted the sensitivity to catch stealthier reconnaissance efforts early. I modified the logic to trigger when <code>event_count &gt; 100</code> and <code>url_original_count_distinct &gt; 50</code>.</p>
<p>For production environments where applications naturally generate higher error volumes, you might consider increasing these values or appending an ES|QL <code>where not</code> clause to exclude known broken links. Finally, I use a <code>where source.ip not in (...)</code> filter to ensure that authorized security tools or personal vulnerability scanners are not accidentally banned by the automated workflow.</p>
<h3>Ingesting Traefik access logs</h3>
<p>To ingest the Traefik access logs into the cluster, I used the default <a href="https://www.elastic.co/docs/reference/integrations/traefik">integration for Traefik</a>. The Elastic Agent collects logs from Traefik servers. This integration writes the ingested logs into the <code>logs-traefik.access-default</code> datastream.</p>
<p><img src="https://www.elastic.co/security-labs/assets/images/detecting-web-server-probing-and-fuzzing/image5.png" alt="" /></p>
<h3>Building a custom ingest pipeline</h3>
<p>The <code>host.name</code> field is crucial for the detection rule I'm using, but the default Traefik integration doesn't populate it. Therefore, a custom ingest pipeline is required to add this field. Since the Traefik integration utilizes a file stream on the Traefik server, I can copy the value from the existing <code>agent.name</code> field to populate <code>host.name</code>.</p>
<p>I specifically use the <code>logs-traefik.access@custom</code> pipeline instead of modifying the main one. Elastic integrations are designed to automatically pick up and run these <code>@custom</code> pipelines right at the end of their processing flow. More importantly, default pipelines get completely overwritten whenever I upgrade an integration. Stashing my logic in the custom pipeline ensures that my field mappings actually survive the next update. The necessary API call to create this pipeline can be executed in the Dev Tools console:</p>
<pre><code class="language-json">PUT _ingest/pipeline/logs-traefik.access@custom
{
  &quot;description&quot;: &quot;copy the agent.name field to the host.name field&quot;,
  &quot;processors&quot;: [
    {
      &quot;set&quot;: {
        &quot;field&quot;: &quot;host.name&quot;,
        &quot;value&quot;: &quot;{{{agent.name}}}&quot;,
        &quot;override&quot;: false,
        &quot;ignore_empty_value&quot;: true,
        &quot;ignore_failure&quot;: true
      }
    }
  ]
}
</code></pre>
<h2>Automated response via Cloudflare workflow</h2>
<p>To move from detection to active defense, we implement a workflow that bridges the gap between our Elastic alerts and the Cloudflare edge. The logic is designed to be efficient: rather than creating a new firewall rule for every single alert, which would quickly hit Cloudflare’s rule limits, the workflow first retrieves the existing blocklist. It then dynamically appends the new offending source IP to that list before pushing the update back to the Cloudflare API. Once the edge is secured, the workflow finishes by acknowledging the alert in Elastic, effectively closing the loop on the incident.</p>
<h3>Prerequisites and token scope</h3>
<p>This process requires both an API key and the Zone ID for the Cloudflare configuration. The API token must possess &quot;Zone WAF edit&quot; privileges to enable the creation of the rule. When generating this token in the Cloudflare dashboard, use the &quot;Create Custom Token&quot; option and set the permissions strictly to <code>Zone -&gt; Zone WAF -&gt; Edit</code><strong>.</strong></p>
<p><img src="https://www.elastic.co/security-labs/assets/images/detecting-web-server-probing-and-fuzzing/image3.png" alt="" /></p>
<p>Once the workflow is configured, it must be assigned as an action to the &quot;Web Server Discovery or Fuzzing Activity&quot; detection rule.</p>
<p>With the prerequisites in place, let's walk through how we build the workflow step-by-step.</p>
<h3>Workflow configuration and triggers</h3>
<p>First, we define the basic metadata. This workflow blocks the IP addresses found in the alerts of the Web Server Discovery or Fuzzing Activity. The workflow is enabled and has a timeout of 30 seconds for the API request. In this case, it's based on an alert, so it runs automatically when a security alert is triggered.</p>
<pre><code># =========================================================================
# Workflow: Block IP at Cloudflare test
# Category: security/response
# =========================================================================
version: '1'
name: Block IP at Cloudflare
enabled: true

triggers:
  - type: alert
</code></pre>
<h3>Constants and authentication</h3>
<p>This section holds the variables for authentication. Remember to substitute the placeholder strings with your actual API token and Zone ID.</p>
<pre><code>consts:
  cloudflare_api: &quot;&lt;cloudflare API&gt;&quot;
  cloudflare_zone: &quot;&lt;cloudflare ZONE&gt;&quot;
</code></pre>
<h3>Step 1: Retrieving the current blocklist</h3>
<p>The sequence checks if the firewall rule already exists. The workflow makes an HTTP GET request to retrieve the existing IP block rule.</p>
<pre><code>steps:
  - name: cloudflare_current_block
    type: http
    with:
      url: &quot;https://api.cloudflare.com/client/v4/zones/{{consts.cloudflare_zone}}/rulesets/phases/http_request_firewall_custom/entrypoint&quot;
      headers:
        Authorization: Bearer {{consts.cloudflare_api}}
      method: GET
    on-failure:
      continue: true
</code></pre>
<h3>Step 2: Updating or creating the firewall rule</h3>
<p>If it exists, the rule gets appended with the IP address otherwise, the rule gets created. The workflow identifies if the &quot;webserver scanning block&quot; description is present. If so, it appends the new IP address to the current list of blocked IP addresses via a PUT request. If not, it falls back to creating a new rule.</p>
<pre><code> - name: cloudflare_block
    type: if
    condition: 'steps.cloudflare_current_block.output.data.result.rules[0].description == &quot;webserver scanning block&quot;'
    steps:
      - name: ip-block-cloudflare_add
        type: http
        with:
          url: &quot;https://api.cloudflare.com/client/v4/zones/{{consts.cloudflare_zone}}/rulesets/phases/http_request_firewall_custom/entrypoint&quot;
          method: PUT
          headers:
            Authorization: Bearer {{consts.cloudflare_api}}
          timeout: 30s
          body: '{ &quot;rules&quot;: [ { &quot;description&quot;: &quot;webserver scanning block&quot;, &quot;expression&quot;: &quot;{{steps.cloudflare_current_block.output.data.result.rules[0].expression}} or (ip.src eq {{event.alerts[0].source.ip}})&quot;, &quot;action&quot;: &quot;block&quot; } ]}'
    else:
      - name: ip-block-cloudflare_new
        type: http
        with:
          url: &quot;https://api.cloudflare.com/client/v4/zones/{{consts.cloudflare_zone}}/rulesets/phases/http_request_firewall_custom/entrypoint&quot;
          method: PUT
          headers:
            Authorization: Bearer {{consts.cloudflare_api}}
          timeout: 30s
          body: '{ &quot;rules&quot;:[ { &quot;description&quot;: &quot;webserver scanning block&quot;, &quot;expression&quot;: &quot;(ip.src eq {{event.alerts[0].source.ip}})&quot;, &quot;action&quot;: &quot;block&quot; } ]}'
    on-failure:
      continue: true
</code></pre>
<h3>Step 3: Acknowledging the alert</h3>
<p>Then the alert gets acknowledged. This step uses the <code>kibana.SetAlertsStatus</code> action to automatically close out the alert in Elastic Security.</p>
<pre><code>  - name: update_alert_status
    type: kibana.SetAlertsStatus
    with:
      status: &quot;acknowledged&quot;
      signal_ids: [&quot;{{event.alerts[0]._id}}&quot;]
</code></pre>
<h3>Step 4: Attaching the Workflow to the Rule</h3>
<p>With the workflow fully built, the final step is to actually attach it to the detection rule so it fires automatically. In the Elastic Security rule settings for the &quot;Web Server Discovery or Fuzzing Activity&quot; rule, I navigate to the <strong>Rule actions</strong> tab and add a new action. From the connector dropdown, I simply select the Cloudflare workflow I just created.</p>
<p><img src="https://www.elastic.co/security-labs/assets/images/detecting-web-server-probing-and-fuzzing/image2.png" alt="" /></p>
<h3>Note on WAF limits</h3>
<p>Because this workflow concatenates IP addresses using an <code>or</code> statement (<code>or (ip.src eq &lt;IP&gt;)</code>), be mindful that Cloudflare has a character limit for custom WAF expressions (typically 4096 characters on standard tiers). In highly targeted environments, this string can eventually hit the limit. For homelabs and small teams, occasionally clearing out this WAF rule manually serves as a healthy reset.</p>
<h2>Testing and Validation</h2>
<p>To verify the pipeline is working end-to-end, we can generate some noise with a standard fuzzing tool. You can simulate a scanning attack against your own homelab using a fuzzing tool like <code>ffuf</code> or <code>gobuster</code>.</p>
<p>Run a quick scan against a non-existent directory on your public-facing Traefik domain:</p>
<pre><code class="language-shell">ffuf -u https://your-domain.com/FUZZ -w /path/to/wordlist.txt
</code></pre>
<p>Once the simulation is running, we can observe the automated defense chain in action. The 404 errors appear almost immediately in the <code>logs-traefik.access-default</code> datastream. Within the polling interval, the ES|QL rule identifies the pattern and generates a new alert in the Elastic Security Alerts page. From there, the workflow takes over: it shifts the alert status to &quot;acknowledged&quot; and pushes the IP block to our Cloudflare WAF rule, effectively neutralizing the scanner at the edge before it can continue its reconnaissance.</p>
<p>You can confirm the block was successful by checking your Cloudflare Dashboard under <code>Security -&gt; WAF -&gt; Custom rules</code>. <em>(Note: Be sure to remove your IP from the Cloudflare rule afterwards so you don't lock yourself out!)</em></p>
<h3>Expanding the defense</h3>
<p>The beauty of this setup is that our Cloudflare workflow isn't limited to just fuzzing detection. Once the automation is built, we can attach it to any Elastic rule that flags suspicious proxy traffic. For instance, we can tie this exact same response action to out-of-the-box rules targeting specific application exploits, like <a href="https://elastic.github.io/detection-rules-explorer/rules/90e4ceab-79a5-4f8e-879b-513cac7fcad9">Web Server Local File Inclusion Activity</a>, <a href="https://elastic.github.io/detection-rules-explorer/rules/45d099b4-a12e-4913-951c-0129f73efb41">Web Server Potential Remote File Inclusion Activity</a> to drop the attacker immediately. It also pairs perfectly with <a href="https://elastic.github.io/detection-rules-explorer/rules/6631a759-4559-4c33-a392-13f146c8bcc4">Potential Spike in Web Server Error Logs</a> and <a href="https://elastic.github.io/detection-rules-explorer/rules/a1b7ffa4-bf80-4bf1-86ad-c3f4dc718b35">Unusual Web User Agent</a> to catch misconfigured scrapers and broader network noise. We build the plumbing once, and suddenly the whole perimeter gets smarter.</p>
<h3>Conclusion</h3>
<p>Wiring Traefik and Cloudflare into Elastic Security is a great way to turn basic access logs into an active defense. Homelab environments are constantly bombarded by automated scanners looking for low-hanging fruit. This automated workflow not only blocks attackers at the edge but also reduces alert fatigue by acknowledging the incidents automatically. It is a practical example of how security orchestration and response can save time while significantly improving your security posture.</p>]]></content:encoded>
            <category>security-labs</category>
            <enclosure url="https://www.elastic.co/security-labs/assets/images/detecting-web-server-probing-and-fuzzing/detecting-web-server-probing-and-fuzzing.webp" length="0" type="image/webp"/>
        </item>
    </channel>
</rss>