Configuring Auditing

Important

Audit logs are disabled by default. To enable this functionality the following setting should be added to the elasticsearch.yml file:

shield.audit.enabled: true

The audit functionality was added to keep track of important events occurring in Elasticsearch, primarily around security concerns. Keeping track and persisting these events is essential for any secured environment and potentially provides evidence for suspicious/malicious activity on the Elasticsearch cluster.

Shield provides two ways to output these events: in a dedicated access.log file stored on the host’s file system, or in an Elasticsearch index on the same or separate cluster. These options are not mutually exclusive. For example, both options can be enabled through an entry in the elasticsearch.yml file:

shield.audit.outputs: [index, logfile]

It is expected that the index output type will be used in conjunction with the logfile output type. This is because the index output type can lose messages if the target index is unavailable. For this reason, it is recommended that, if auditing is enabled, then the logfile output type should be used as an official record of events. The index output type can be enabled as a convenience to allow historical browsing of events.

Please also note that, because audit events are batched together before being indexed, they may not appear immediately. Please refer to the shield.audit.index.flush_interval setting below for instructions on how to modify the frequency with which batched events are flushed.

Log Entry Types

Each audit related event that occurs is represented by a single log entry of a specific type (the type represents the type of the event that occurred). Here are the possible log entry types:

  • anonymous_access_denied is logged when the request is denied due to missing authentication token.
  • authentication_failed is logged when the authentication token cannot be matched to a known user.
  • authentication_failed [<realm>] is logged for every realm that fails to present a valid authentication token. The value of <realm> is the realm type.
  • access_denied is logged when an authenticated user attempts an action the user does not have the privilege to perform.
  • access_granted is logged when an authenticated user attempts an action the user has the correct privilege to perform. In TRACE level all system (internal) actions are logged as well (in all other level they’re not logged to avoid cluttering of the logs.
  • tampered_request is logged when the request was detected to be tampered (typically relates to search/scroll requests when the scroll id is believed to be tampered)
  • connection_granted is logged when an incoming tcp connection has passed the ip filtering for a specific profile
  • connection_denied is logged when an incoming tcp connection did not pass the ip filtering for a specific profile

To avoid needless proliferation of log entries, Shield enables you to control what entry types should be logged. This can be done by setting the logging level. The following table lists the log entry types that will be logged for each of the possible log levels:

Table 6. Log Entry Types and Levels

Log Level Entry Type

ERROR

authentication_failed, access_denied, tampered_request, connection_denied

WARN

authentication_failed, access_denied, tampered_request, connection_denied, anonymous_access_denied

INFO

authentication_failed, access_denied, tampered_request, connection_denied, anonymous_access_denied, access_granted

DEBUG

(doesn’t output additional entry types beyond INFO, but extends the information emitted for each entry (see Log Entry Format below)

TRACE

authentication_failed, access_denied, tampered_request, connection_denied, anonymous_access_denied, access_granted, connection_granted, authentication_failed [<realm>]. In addition, internal system requests (self-management requests triggered by Elasticsearch itself) will also be logged for access_granted entry type.


Log Entry Format

As mentioned above, every log entry represents an event that occurred in the system. As such, each entry is associated with a timestamp (at which the event occurred), the component/layer the event is associated with and the entry/event type. In addition, every log entry (depending ot its type) carries addition information about the event.

The format of a log entry is shown below:

[<timestamp>] [<local_node_info>] [<layer>] [<entry_type>] <attribute_list>

Where:

  • <timestamp> - the timestamp of the entries (in the format configured in logging.yml as shown above)
  • <local_node_info> - additional information about the local node that this log entry is printed from (the table below shows how this information can be controlled via settings)
  • <layer> - the layer from which this entry relates to. Can be either rest, transport or ip_filter
  • <entry_type> - the type of the entry as discussed above. Can be either anonymous_access_denied, authentication_failed, access_denied, access_granted, connection_granted, connection_denied.
  • <attribute_list> - A comma-separated list of attribute carrying data relevant to the occurred event (formatted as attr1=[val1], attr2=[val2],...)

Table 7. Local Node Info Settings

Name Default Description

shield.audit.logfile.prefix.emit_node_name

true

When set to true, the local node’s name will be emitted

shield.audit.logfile.prefix.emit_node_host_address

false

When set to true, the local node’s IP address will be emitted

shield.audit.logfile.prefix.emit_node_host_name

false

When set to true, the local node’s host name will be emitted


The following tables describe the possible attributes each entry type can carry (the attributes that will be available depend on the configured log level):

Table 8. [rest] [anonymous_access_denied] attributes

Attribute Minimum Log Level Description

origin_address

WARN

The address the rest request origins from

uri

WARN

The REST endpoint URI

request_body

DEBUG

The body of the request


Table 9. [rest] [authentication_failed] attributes

Attribute Minimum Log Level Description

origin_address

ERROR

The address the rest request origins from

principal

ERROR

The principal (username) that failed to authenticate

uri

ERROR

The REST endpoint URI

request_body

DEBUG

The body of the request

realm

TRACE

The realm that failed to authenticate the user. NOTE: A separate entry will be printed for each of the consulted realms


Table 10. [transport] [anonymous_access_denied] attributes

Attribute Minimum Log Level Description

origin_type

WARN

The type of the origin the request originated from. Can be either rest (request was originated from a rest API request), transport (request received on the transport channel), local_node (the local node issued the request)

origin_address

WARN

The address the request origins from

action

WARN

The name of the action that was executed

request

DEBUG

The type of the request that was executed

indices

WARN

A comma-separated list of indices this request relates to (when applicable)


Table 11. [transport] [authentication_failed] attributes

Attribute Minimum Log Level Description

origin_type

ERROR

The type of the origin the request originated from. Can be either rest (request was originated from a rest API request), transport (request received on the transport channel), local_node (the local node issued the request)

origin_address

ERROR

The address the request origins from

principal

ERROR

The principal (username) that failed to authenticate

action

ERROR

The name of the action that was executed

request

DEBUG

The type of the request that was executed

indices

ERROR

A comma-separated list of indices this request relates to (when applicable)

realm

TRACE

The realm that failed to authenticate the user. NOTE: A separate entry will be printed for each of the consulted realms


Table 12. [transport] [access_granted] attributes

Attribute Minimum Log Level Description

origin_type

INFO

The type of the origin the request originated from. Can be either rest (request was originated from a rest API request), transport (request received on the transport channel), local_node (the local node issued the request)

origin_address

INFO

The address the request origins from

principal

INFO

The principal (username) that failed to authenticate

action

INFO

The name of the action that was executed

request

DEBUG

The type of the request that was executed

indices

INFO

A comma-separated list of indices this request relates to (when applicable)


Table 13. [transport] [access_denied] attributes

Attribute Minimum Log Level Description

origin_type

ERROR

The type of the origin the request originated from. Can be either rest (request was originated from a rest API request), transport (request received on the transport channel), local_node (the local node issued the request)

origin_address

ERROR

The address the request origins from

principal

ERROR

The principal (username) that failed to authenticate

action

ERROR

The name of the action that was executed

request

DEBUG

The type of the request that was executed

indices

ERROR

A comma-separated list of indices this request relates to (when applicable)


Table 14. [transport] [tampered_request] attributes

Attribute Minimum Log Level Description

origin_type

ERROR

The type of the origin the request originated from. Can be either rest (request was originated from a rest API request), transport (request received on the transport channel), local_node (the local node issued the request)

origin_address

ERROR

The address the request origins from

principal

ERROR

The principal (username) that failed to authenticate

action

ERROR

The name of the action that was executed

request

DEBUG

The type of the request that was executed

indices

ERROR

A comma-separated list of indices this request relates to (when applicable)


Table 15. [ip_filter] [connection_granted] attributes

Attribute Minimum Log Level Description

origin_address

TRACE

The address the request origins from

transport_profile

TRACE

The principal (username) that failed to authenticate

rule

TRACE

The IP filtering rule that granted the request


Table 16. [ip_filter] [connection_denied] attributes

Attribute Minimum Log Level Description

origin_address

ERROR

The address the request origins from

transport_profile

ERROR

The principal (username) that failed to authenticate

rule

ERROR

The IP filtering rule that denied the request


Audit Logs Settings

As mentioned above, the audit logs are configured in the logging.yml file located in CONFIG_DIR/shield. The following snippet shows the default logging configuration:

Default logging.yml File. 

logger:
  shield.audit.logfile: INFO, access_log

additivity:
  shield.audit.logfile: false

appender:

  access_log:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}-access.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}] %m%n"

As can be seen above, by default audit information is appended to the access.log file located in the standard Elasticsearch logs directory (typically located at $ES_HOME/logs).

Storing Audit Logs in an Elasticsearch Index

It is possible to store audit logs in an Elasticsearch index. This index can be either on the same cluster, or on a different cluster (see below). Several settings in elasticsearch.yml control this behavior.

Table 17. audit log indexing configuration

Attribute Default Setting Description

shield.audit.outputs

logfile

Must be set to index or [index, logfile] to enable

shield.audit.index.bulk_size

1000

Controls how many audit events will be batched into a single write

shield.audit.index.flush_interval

1s

Controls how often to flush buffered events into the index

shield.audit.index.rollover

daily

Controls how often to roll over to a new index: hourly, daily, weekly, monthly.

shield.audit.index.events.include

anonymous_access_denied, authentication_failed, access_granted, access_denied, tampered_request, connection_granted, connection_denied

The audit events to be indexed. Valid values are anonymous_access_denied, authentication_failed, access_granted, access_denied, tampered_request, connection_granted, connection_denied, system_access_granted. _all is a special value that includes all types.

shield.audit.index.events.exclude

system_access_granted

The audit events to exclude from indexing. By default, system_access_granted events are excluded; enabling these events results in every internal node communication being indexed, which will make the index size much larger.


audit index settings. The settings for the index that the events are stored in, can also be configured. The index settings should be placed under the shield.audit.index.settings namespace. For example, the following sets the number of shards and replicas to 1 for the audit indices:

shield.audit.index.settings:
  index:
    number_of_shards: 1
    number_of_replicas: 1

Forwarding Audit Logs to a Remote Cluster

To have audit events stored into a remote Elasticsearch cluster, the additional following options are available.

Table 18. remote audit log indexing configuration

Attribute Default Setting Description

shield.audit.index.client.hosts

None

Comma separated list of host:port pairs. These hosts should be nodes in the cluster to which you want to index.

shield.audit.index.client.cluster.name

None

The name of the remote cluster.

shield.audit.index.client.shield.user

None

The username:password pair used to authenticate with the remote cluster.


Additional settings may be passed to the remote client by placing them under the shield.audit.index.client namespace. For example, to allow the remote client to discover all of the nodes in the remote cluster you could set the client.transport.sniff option.

shield.audit.index.client.transport.sniff: true