Segfault from Sensitive Process Detected

edit
IMPORTANT: This documentation is no longer updated. Refer to Elastic's version policy and the latest documentation.

Segfault from Sensitive Process Detected

edit

Monitors kernel logs for segfault messages from sensitive processes. A segfault, or segmentation fault, is an error that occurs when a program tries to access a memory location that it’s not allowed to access, typically leading to program termination. A segfault can be an indication of malicious behavior if it results from attempts to exploit buffer overflows, inject shared objects, or other vulnerabilities in software to execute arbitrary code or disrupt its normal operation.

Rule type: query

Rule indices:

  • logs-system.syslog-*
  • filebeat-*

Severity: medium

Risk score: 47

Runs every: 5m

Searches indices from: now-9m (Date Math format, see also Additional look-back time)

Maximum alerts per execution: 100

References: None

Tags:

  • Domain: Endpoint
  • OS: Linux
  • Use Case: Threat Detection
  • Tactic: Credential Access
  • Tactic: Execution
  • Resources: Investigation Guide

Version: 1

Rule authors:

  • Elastic

Rule license: Elastic License v2

Investigation guide

edit
## Triage and analysis

Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

Investigating Segfault from Sensitive Process Detected

This alert flags a Linux kernel-reported segmentation fault in a sensitive service or authentication-related process, which matters because unexpected crashes in these programs often mark memory corruption, exploit attempts, or deliberate disruption around privileged execution and credential handling. An attacker might feed a crafted request to sshd, sudo, pkexec, or a web daemon to trigger a crash while exploiting a buffer overflow or forcing a malicious library load to gain code execution or access secrets.

Possible investigation steps

  • Correlate the crash timestamp with application, authentication, and reverse-proxy logs plus inbound connections to identify the exact user action, request, or source IP that immediately preceded the fault.
  • Determine whether the executable and loaded libraries changed recently by comparing package version, file hashes, ownership, and recent writes in system paths, and note any recent upgrades that could explain a benign stability issue.
  • Collect the core dump, journald entries, and surrounding kernel messages to extract the faulting address, signal details, stack trace, and library references, which can help distinguish a known software bug from memory-corruption or injection activity.
  • Review adjacent endpoint telemetry on the same host for exploitation indicators such as abnormal child processes, privilege-escalation attempts, ptrace activity, unexpected environment manipulation, dropped shared objects, or new outbound connections after the crash.
  • If the crashed service handles authentication or privileged actions, scope impact by identifying failed or successful logins, credential access attempts, service restarts, and user disruption, then isolate and patch the host if crashes recur or align with suspicious activity.

False positive analysis

  • A legitimate package upgrade or service restart can expose a software bug in a sensitive daemon such as sshd, nginx, or sudo; verify by checking for recent version or library changes, planned maintenance, and the absence of suspicious requests or follow-on process activity around the crash time.
  • An approved configuration change or normal user action can trigger an edge-case parsing error that crashes services such as httpd, rsyslogd, or openvpn; verify by reviewing recent config edits and service logs to confirm the segfault aligns with the authorized change and does not coincide with anomalous authentication or network events.

Response and remediation

  • Isolate the affected Linux host from the network, stop the crashing sensitive service, and block the source IPs, URLs, or client requests that immediately preceded the segfault to prevent additional exploitation.
  • Preserve the core dump, journald history, and the crashing binary for evidence, then escalate to incident response immediately if the crash was followed by root shell activity, successful sudo or pkexec use, access to /etc/shadow or credential stores, or similar segfaults on multiple hosts.
  • Remove attacker persistence by deleting unauthorized shared objects, LD_PRELOAD entries, cron jobs, systemd units, startup scripts, web shells, modified SSH authorized_keys files, and any trojanized copies of the affected executable or its libraries.
  • Restore the system to a known-good state by rebuilding from a trusted image or reinstalling affected packages from signed repositories, validate file integrity before reconnecting it, and rotate passwords, SSH keys, API tokens, and service secrets that may have been exposed on the host.
  • Harden the environment by patching the vulnerable service and dependent libraries, enforcing SELinux or AppArmor and least-privilege service accounts, restricting who can read core dumps, and adding detections for repeated crashes, unexpected library loads, and child processes spawned by sensitive services.

Rule query

edit
host.os.type:linux and event.dataset:system.syslog and process.name:kernel and
message:(
  segfault and (
    agetty or apache2 or atd or auditbeat or auditd or beacon-chain or besu or chage or
    chfn or chsh or clef or cron or crond or dbus-broker or dbus-daemon or dnsmasq or
    elastic-agent or erigon or ethrex or ethsigner or geth or getty or gpasswd or
    grandine or httpd or krb5_child or ldap_child or lighthouse or lodestar or login or
    logrotate or named or nethermind or newgrp or nginx or nslcd or op-batcher or
    op-challenger or op-conductor or op-geth or op-node or op-proposer or openvpn or
    osqueryd or passwd or pkexec or polkitd or proftpd or prysm or reth or rsyslogd or
    smbd or ssh or sshd or sssd or sssd_nss or sssd_pam or su or sudo or sudoedit or
    systemd-logind or teku or unix_chkpwd or vsftpd or web3signer
  )
)

Framework: MITRE ATT&CKTM