Azure moduleedit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

The azure module retrieves different types of log data from Azure. There are several requirements before using the module since the logs will actually be read from azure event hubs.

The module contains the following filesets:

activitylogs
Will retrieve azure activity logs. Control-plane events on Azure Resource Manager resources. Activity logs provide insight into the operations that were performed on resources in your subscription.
signinlogs
Will retrieve azure Active Directory sign-in logs. The sign-ins report provides information about the usage of managed applications and user sign-in activities.
auditlogs
Will retrieve azure Active Directory audit logs. The audit logs provide traceability through logs for all changes done by various features within Azure AD. Examples of audit logs include changes made to any resources within Azure AD like adding or removing users, apps, groups, roles and policies.

Module configurationedit

- module: azure
  activitylogs:
    enabled: true
    var:
      eventhub: "insights-operational-logs"
      consumer_group: "$Default"
      connection_string: ""
      storage_account: ""
      storage_account_key: ""
      resource_manager_endpoint: ""

   auditlogs:
     enabled: false
     var:
       eventhub: "insights-logs-auditlogs"
       consumer_group: "$Default"
       connection_string: ""
       storage_account: ""
       storage_account_key: ""
       resource_manager_endpoint: ""

   signinlogs:
     enabled: false
     var:
       eventhub: ["insights-logs-signinlogs"]
       consumer_group: "$Default"
       connection_string: ""
       storage_account: ""
       storage_account_key: ""
       resource_manager_endpoint: ""
eventhub
[]string Is a fully managed, real-time data ingestion service. Default value insights-operational-logs
consumer_group
string The publish/subscribe mechanism of Event Hubs is enabled through consumer groups. A consumer group is a view (state, position, or offset) of an entire event hub. Consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at their own pace and with their own offsets. Default value: $Default
connection_string
string The connection string required to communicate with Event Hubs, steps here https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-get-connection-string.

A Blob Storage account is required in order to store/retrieve/update the offset or state of the eventhub messages. This means that after stopping the filebeat azure module it can start back up at the spot that it stopped processing messages.

storage_account
string The name of the storage account the state/offsets will be stored and updated.
storage_account_key
string The storage account key, this key will be used to authorize access to data in your storage account.
resource_manager_endpoint
string Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different azure environment. Ex: https://management.chinacloudapi.cn/ for azure ChinaCloud https://management.microsoftazure.de/ for azure GermanCloud https://management.azure.com/ for azure PublicCloud https://management.usgovcloudapi.net/ for azure USGovernmentCloud Users can also use this in case of a Hybrid Cloud model, where one may define their own endpoints.

When you run the module, it performs a few tasks under the hood:

  • Sets the default paths to the log files (but don’t worry, you can override the defaults)
  • Makes sure each multiline log event gets sent as a single event
  • Uses ingest node to parse and process the log lines, shaping the data into a structure suitable for visualizing in Kibana

Read the quick start to learn how to set up and run modules.

Compatibilityedit

TODO: document with what versions of the software is this tested

Dashboardsedit

The azure module comes with several predefined dashboards for general cloud overview, user activity and alerts. For example:

filebeat azure overview

Fieldsedit

For a description of each field in the module, see the exported fields section.