While the benefits of using serverless functions are manifold, it is also necessary to make them observable in the wider end-to-end microservices architecture context.

Elastic Observability (APM) for Azure Functions: The architecture

Elastic Observability 8.7 introduced distributed tracing for Microsoft Azure Functions — available for the Elastic APM Agents for .NET, Node.js, and Python. Auto-instrumentation of HTTP requests is supported out-of-the-box, enabling the detection of performance bottlenecks and sources of errors.

The key components of the solution for observing Azure Functions are:

1. The Elastic APM Agent for the relevant language
2. Elastic Observability

The APM server validates and processes incoming events from individual APM Agents and transforms them into Elasticsearch documents. The APM Agent provides auto-instrumentation capabilities for the application being observed. The Node.js APM Agent can trace function invocations in an Azure Functions app.

Setting up Elastic APM for Azure Functions

To demonstrate the setup and usage of Elastic APM, we will use a sample Node.js application.

Application overview

The Node.js application has two HTTP-triggered functions named "Hello" and "Goodbye." Once deployed, they can be called as follows, and tracing data will be sent to the configured Elastic Observability deployment.

curl -i https://<APP_NAME>.azurewebsites.net/api/hello
curl -i https://<APP_NAME>.azurewebsites.net/api/goodbye

Setup

Step 0. Prerequisites

To run the sample application, you will need:

• An installation of Node.js (v14 or later)
• Access to an Azure subscription with an appropriate role to create resources
• The Azure CLI (az) logged into an Azure subscription

1. Use az login to login
2. See the output of az account show

• The Azure Functions Core Tools (func) (func --version should show a 4.x version)
• An Elastic Observability deployment to which monitoring data will be sent

1. The simplest way to get started with Elastic APM Microsoft Azure is through Elastic Cloud. Get started with Elastic Cloud on Azure Marketplace or sign up for a trial on Elastic Cloud.

• The APM server URL (serverUrl) and secret token (secretToken) from your Elastic stack deployment for configuration below

1. How to get the serverUrl and secretToken documentation

Step 1. Clone the sample application repo and install dependencies

git clone https://github.com/elastic/azure-functions-apm-nodejs-sample-app.git
cd azure-functions-apm-nodejs-sample-app
npm install

Step 2. Deploy the Azure Function App
Caution icon! Deploying a function app to Azure can incur costs. The following setup uses the free tier of Azure Functions. Step 5 covers the clean-up of resources.

Step 2.1
To avoid name collisions with others that have independently run this demo, we need a short unique identifier for some resource names that need to be globally unique. We'll call it the DEMO_ID. You can run the following to generate one and save it to DEMO_ID and the "demo-id" file.

if [[ ! -f demo-id ]]; then node -e 'console.log(crypto.randomBytes(3).toString("hex"))' >demo-id; fi
export DEMO_ID=$(cat demo-id)
echo $DEMO_ID

Step 2.2
Before you can deploy to Azure, you will need to create some Azure resources: a Resource Group, Storage Account, and the Function App. For this demo, you can use the following commands. (See this Azure docs section for more details.)

REGION=westus2   # Or use another region listed in 'az account list-locations'.
az group create --name "AzureFnElasticApmNodeSample-rg" --location "$REGION"
az storage account create --name "eapmdemostor${DEMO_ID}" --location "$REGION" \
    --resource-group "AzureFnElasticApmNodeSample-rg" --sku Standard_LRS
az functionapp create --name "azure-functions-apm-nodejs-sample-app-${DEMO_ID}" \
    --resource-group "AzureFnElasticApmNodeSample-rg" \
    --consumption-plan-location "$REGION" --runtime node --runtime-version 18 \
    --functions-version 4 --storage-account "eapmdemostor${DEMO_ID}"

Step 2.3
Next, configure your Function App with the APM server URL and secret token for your Elastic deployment. This can be done in the Azure Portal or with the az CLI.

In the Azure portal, browse to your Function App, then its Application Settings (Azure user guide). You'll need to add two settings:

First set your APM URL and token.

export ELASTIC_APM_SERVER_URL="<your serverUrl>"
export ELASTIC_APM_SECRET_TOKEN="<your secretToken>"

Or you can use the az functionapp config appsettings set … CLI command as follows:

az functionapp config appsettings set \
  -g "AzureFnElasticApmNodeSample-rg" -n "azure-functions-apm-nodejs-sample-app-${DEMO_ID}" \
  --settings "ELASTIC_APM_SERVER_URL=${ELASTIC_APM_SERVER_URL}"
az functionapp config appsettings set \
  -g "AzureFnElasticApmNodeSample-rg" -n "azure-functions-apm-nodejs-sample-app-${DEMO_ID}" \
  --settings "ELASTIC_APM_SECRET_TOKEN=${ELASTIC_APM_SECRET_TOKEN}"

The ELASTIC_APM_SERVER_URL and ELASTIC_APM_SECRET_TOKEN are set in Azure function’s settings for the app and used by the Elastic APM Agent. This is initiated by the initapm.js file, which starts the Elastic APM agent with:

require("elastic-apm-node").start();

When you log in to Azure and look at the function’s configuration, you will see them set:

Step 2.4
Now you can publish your app. (Re-run this command every time you make a code change.)

func azure functionapp publish "azure-functions-apm-nodejs-sample-app-${DEMO_ID}"

You should log in to Azure to see the function running.

Step 3. Try it out

% curl https://azure-functions-apm-nodejs-sample-app-${DEMO_ID}.azurewebsites.net/api/Hello
{"message":"Hello."}
% curl https://azure-functions-apm-nodejs-sample-app-${DEMO_ID}.azurewebsites.net/api/Goodbye
{"message":"Goodbye."}

In a few moments, the APM app in your Elastic deployment will show tracing data for your Azure Function app.

Step 4. Apply some load to your app
To get some more interesting data, you can run the following to generate some load on your deployed function app:

npm run loadgen

This uses the autocannon node package to generate some light load (2 concurrent users, each calling at 5 requests/s for 60s) on the "Goodbye" function.

Step 5. Clean up resources
If you deployed to Azure, you should make sure to delete any resources so you don't incur any costs.

az group delete --name "AzureFnElasticApmNodeSample-rg"

Analyzing Azure Function APM data in Elastic

Once you have successfully set up the sample application and started generating load, you should see APM data appearing in the Elastic Observability APM Services capability.

Service map

With the default setup, you will see two services in the APM Service map.

The main function: azure-functions-apm-nodejs-sample-app

And the end point where your function is accessible: azure-functions-apm-nodejs-sample-app-ec7d4c.azurewebsites.net

You will see that there is a connection between the two as your application is taking requests and answering through the endpoint.

From the APM Service map you can further investigate the function, analyze traces, look at logs, and more.

Service details

When we dive into the details, we can see several items.

• Latency for the recent load we ran against the application
• Transactions (Goodbye and Hello)
• Average throughput
• And more

Transaction details

We can see transaction details.

An individual trace shows us that the "Goodbye" function calls the "Hello" function in the same function app before returning:

Machine learning based latency correlation

As we’ve mentioned in other blogs, we can also correlate issues such as higher than normal latency. Since we see a spike at 1s, we run the embedded latency correlation, which uses machine learning to help analyze the potential impacting component by analyzing logs, metrics, and traces.

The correlation indicated there is a potential cause (25%) due to the host sending the load (my machine).

Cold start detection

Also, we can see the impact a cold start can have on the latency of a request:

Summary

Elastic Observability provides real-time monitoring of Azure Functions in your production environment for a broad range of use cases. Curated dashboards assist DevOps teams in performing root cause analysis for performance bottlenecks and errors. SRE teams can quickly view upstream and downstream dependencies, as well as perform analyses in the context of distributed microservices architecture.

Learn more

To learn how to add the Elastic APM Agent to an existing Node.js Azure Function app, read Monitoring Node.js Azure Functions. Additional resources include:

• How to deploy and manage Elastic Observability on Microsoft Azure
• Elastic APM Quickstart