Azure Functions

Get metrics and logs from Azure Functions


Version	0.7.0 (View all)
Compatible Kibana version(s)	8.13.0 or higher
Supported Serverless project types What's this?	Security Observability
Subscription level What's this?	Basic
Level of support What's this?	Elastic

The Azure Functions integration allows you to monitor Azure Functions. Azure Functions is an event-driven, serverless compute platform that helps you develop more efficiently using the programming language of your choice. Triggers cause a function to run. A trigger defines how a function is invoked and a function must have exactly one trigger.

Use this integration to build web APIs, respond to database changes, process IoT streams, manage message queues, and more. Refer common Azure Functions scenarios for more information.

Data streams

The Azure Functions integration contains two data streams: Function App Logs and Metrics

Logs

Supported log categories:

Log Category	Description
Functionapplogs	Function app logs.

Requirements and setup

Refer to the Azure Logs page for more information about setting up and using this integration.

Configuration options

eventhub : string An Event Hub is a fully managed, real-time data ingestion service. Elastic recommends using only letters, numbers, and the hyphen (-) character for Event Hub names to maximize compatibility. You can use existing Event Hubs having underscores (_) in the Event Hub name; in this case, the integration will replace underscores with hyphens (-) when it uses the Event Hub name to create dependent Azure resources behind the scenes (e.g., the storage account container to store Event Hub consumer offsets). Elastic also recommends using a separate event hub for each log type as the field mappings of each log type differ. Default value insights-operational-logs.

consumer_group : string The publish/subscribe mechanism of Event Hubs is enabled through consumer groups. A consumer group is a view (state, position, or offset) of an entire event hub. Consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at their own pace and with their own offsets. Default value: $Default

connection_string : string The connection string is required to communicate with Event Hubs, see steps here.

A Blob Storage account is required in order to store/retrieve/update the offset or state of the eventhub messages. This means that after stopping the Azure logs package it can start back up at the spot that it stopped processing messages.

storage_account : string The name of the storage account where the state/offsets will be stored and updated.

storage_account_key : string The storage account key, this key will be used to authorize access to data in your storage account.

storage_account_container : string The storage account container where the integration stores the checkpoint data for the consumer group. It is an advanced option to use with extreme care. You MUST use a dedicated storage account container for each Azure log type (activity, sign-in, audit logs, and others). DO NOT REUSE the same container name for more than one Azure log type. See Container Names for details on naming rules from Microsoft. The integration generates a default container name if not specified.

resource_manager_endpoint : string Optional, by default we are using the Azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different Azure environment.

Resource manager endpoints:

# Azure ChinaCloud
https://management.chinacloudapi.cn/

# Azure GermanCloud
https://management.microsoftazure.de/

# Azure PublicCloud 
https://management.azure.com/

# Azure USGovernmentCloud
https://management.usgovcloudapi.net/

An example event for functionapplogs looks as following:

{
    "@timestamp": "2023-05-23T20:11:59.000Z",
    "azure": {
        "category": "FunctionAppLogs",
        "function": {
            "app_name": "test-function",
            "category": "Function.hello",
            "event_name": "FunctionStarted",
            "host_instance_id": "bb84c437-4c26-4d0b-a06d-7fc2f16976e3",
            "host_version": "4.19.2.2",
            "invocation_id": "d878e365-b3d6-4796-9292-7500acd0c677",
            "level": "Information",
            "level_id": 2,
            "message": "Executing Functions.hello (Reason=This function was programmatically called via the host APIs., Id=d878e365-b3d6-4796-9292-7500acd0c677)",
            "name": "Functions.hello",
            "process_id": 67,
            "role_instance": "54108609-638204200593759681"
        },
        "operation_name": "Microsoft.Web/sites/functions/log",
        "resource": {
            "group": "TEST-RG",
            "id": "/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/TEST-RG/PROVIDERS/MICROSOFT.WEB/SITES/TEST-FUNCTION",
            "name": "TEST-FUNCTION",
            "provider": "MICROSOFT.WEB/SITES"
        },
        "subscription_id": "12CABCB4-86E8-404F-A3D2-1DC9982F45CA"
    },
    "cloud": {
        "account": {
            "id": "12CABCB4-86E8-404F-A3D2-1DC9982F45CA"
        },
        "provider": "azure"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "event": {
        "original": "{\"time\":\"2023-05-23T20:11:59Z\",\"resourceId\":\"/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/TEST-RG/PROVIDERS/MICROSOFT.WEB/SITES/TEST-FUNCTION\",\"category\":\"FunctionAppLogs\",\"operationName\":\"Microsoft.Web/sites/functions/log\",\"level\":\"Informational\",\"location\":\"East US\",\"properties\":{\"appName\":\"test-function\",\"roleInstance\":\"54108609-638204200593759681\",\"message\":\"Executing Functions.hello (Reason=This function was programmatically called via the host APIs., Id=d878e365-b3d6-4796-9292-7500acd0c677)\",\"category\":\"Function.hello\",\"hostVersion\":\"4.19.2.2\",\"functionInvocationId\":\"d878e365-b3d6-4796-9292-7500acd0c677\",\"functionName\":\"Functions.hello\",\"hostInstanceId\":\"bb84c437-4c26-4d0b-a06d-7fc2f16976e3\",\"level\":\"Information\",\"levelId\":2,\"processId\":67,\"eventId\":1,\"eventName\":\"FunctionStarted\"}}"
    },
    "observer": {
        "product": "Azure Functions",
        "type": "functions",
        "vendor": "Azure"
    },
    "tags": [
        "preserve_original_event"
    ]
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields

Field	Description	Type
@timestamp	Event timestamp.	date
azure.category	The log category name.	keyword
azure.function.app_name	The Function application name.	keyword
azure.function.category	The category of the operation.	keyword
azure.function.event_id	The event ID.	long
azure.function.event_name	The event name.	keyword
azure.function.exception_details	The exception details. This includes the exception type, message, and stack trace.	match_only_text
azure.function.exception_message	The exception message.	match_only_text
azure.function.exception_type	The exception type.	keyword
azure.function.host_instance_id	The host instance ID.	keyword
azure.function.host_version	The Functions host version.	keyword
azure.function.invocation_id	The invocation ID that logged the message.	keyword
azure.function.level	The log level. Valid values are Trace, Debug, Information, Warning, Error, or Critical.	keyword
azure.function.level_id	The integer value of the log level. Valid values are 0 (Trace), 1 (Debug), 2 (Information), 3 (Warning), 4 (Error), or 5 (Critical).	long
azure.function.message	The log message.	keyword
azure.function.name	The name of the function that logged the message.	keyword
azure.function.process_id	The process ID.	long
azure.function.role_instance	The role instance ID.	keyword
azure.operation_name	The operation name.	keyword
azure.resource.group	Azure Resource group	keyword
azure.resource.id	Resource ID	keyword
azure.resource.name	Name	keyword
azure.resource.provider	Resource type/namespace	keyword
azure.subscription_id	Azure subscription ID	keyword
azure.tenant_id	tenant ID	keyword
data_stream.dataset	Data stream dataset name.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
dataset.name	Dataset name.	constant_keyword
dataset.namespace	Dataset namespace.	constant_keyword
dataset.type	Dataset type.	constant_keyword

Metrics

Metrics give you insight into the performance of your Azure Function Apps. The integration includes an out-of-the-box dashboard for visualising the monitoring data generated by apps hosted in Azure Functions.

Requirements

To use this integration you will need:

Azure App Registration: You need to set up an Azure App Registration to allow the Agent to access the Azure APIs. The App Registration requires the Monitoring Reader role to access to be able to collect metrics from Function Apps. See more details in the Setup section.
Elasticsearch and Kibana: You need Elasticsearch to store and search your data and Kibana to visualize and manage it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, the Native Azure Integration, or self-manage the Elastic Stack on your hardware.

Setup

         ┌────────────────────┐       ┌─────────┐       ┌─-─────────────────────┐
         │                    │       │         │       │    azure.functions    │
         │     Azure APIs     │──────▶│  Agent  │──────▶│    <<data stream>>    │
         │                    │       │         │       │                       │
         └────────────────────┘       └─────────┘       └───-───────────────────┘

Elastic Agent needs an App Registration to access Azure on your behalf to collect data using the Azure REST APIs. App Registrations are required to access Azure APIs programmatically.

To start collecting data with this integration, you need to:

Set up a new Azure app registration by registering an app, adding credentials, and assigning an appropriate role.
Specify integration settings in Kibana, which will determine how the integration will access the Azure APIs.

Register a new app

To create a new app registration:

Sign in to the Azure Portal.
Search for and select Microsoft Entra ID.
Under Manage, select App registrations > New registration.
Enter a display Name for your application (for example, "elastic-agent").
Specify who can use the application.
Don't enter anything for Redirect URI. This is optional and the agent doesn't use it.
Select Register to complete the initial app registration.

Take note of the Application (client) ID, which you will use later when specifying the Client ID in the integration settings.

Add credentials

Credentials allow your application to access Azure APIs and authenticate itself, requiring no interaction from a user at runtime.

This integration uses Client Secrets to prove its identity.

In the Azure Portal, select the application you created in the previous section.
Select Certificates & secrets > Client secrets > New client secret.
Add a description (for example, "Elastic Agent client secrets").
Select an expiration for the secret or specify a custom lifetime.
Select Add.

Take note of the content in the Value column in the Client secrets table, which you will use later when specifying a Client Secret in the integration settings. This secret value is never displayed again after you leave this page. Record the secret's value in a safe place.

Assign role

In the Azure Portal, search for and select Subscriptions.
Select the subscription to assign the application.
Select Access control (IAM).
Select Add > Add role assignment to open the Add role assignment page.
In the Role tab, search and select the role Monitoring Reader.
Select the Next button to move to the Members tab.
Select Assign access to > User, group, or service principal, and select Select members. This page does not display Azure AD applications in the available options by default.
To find your application, search by name (for example, "elastic-agent") and select it from the list.
Click the Select button.
Then click the Review + assign button.

Take note of the following values, which you will use later when specifying settings.

Subscription ID: use the content of the "Subscription ID" you selected.
Tenant ID: use the "Tenant ID" from the Azure Active Directory you use.

Your App Registration is now ready to be used with the Elastic Agent.

Additional Resources

If you want to learn more about this process, you can read these two general guides from Microsoft:

Main options

The settings' main section contains all the options needed to access the Azure APIs and collect the Azure Functions metrics data. You will now use all the values from App registration including:

Client ID string: The unique identifier of the App Registration (sometimes referred to as Application ID).
Client Secret string: The client secret for authentication.
Subscription ID string: The unique identifier for the Azure subscription. You can provide just one subscription ID. The Agent uses this ID to access Azure APIs.
Tenant ID string: The unique identifier of the Azure Active Directory's Tenant ID.

Advanced options

There are two additional advanced options:

Resource Manager Endpoint string: Optional. By default, the integration uses the Azure public environment. To override, users can provide a specific resource manager endpoint to use a different Azure environment.

Examples:

https://management.chinacloudapi.cn for Azure ChinaCloud
https://management.microsoftazure.de for Azure GermanCloud
https://management.azure.com for Azure PublicCloud
https://management.usgovcloudapi.net for Azure USGovernmentCloud

Active Directory Endpoint string: Optional. By default, the integration uses the associated Active Directory Endpoint. To override, users can provide a specific active directory endpoint to use a different Azure environment.