Microsoft Exchange Online Message Trace

Serverless Observability Serverless Security Stack 9.1.4

This integration is for Microsoft Exchange Online Message Trace logs. It includes the following datasets for receiving logs over the Microsoft Exchange Online Message Trace API or read from a file:

• log dataset: supports Microsoft Exchange Online Message Trace logs.

Microsoft has announced the deprecation on March 18th, 2026 of the legacy Message Trace API support in the Reporting Webservice, which is used by this integration.

However, that legacy API is already unavailable to new Microsoft tenants, and the replacement Graph-based Message Trace API is not yet available. It is tentatively expected in December 2025.

For the new Message Trace experience, there is an updated PowerShell cmdlet, Get-MessageTraceV2, in General Availability since June 3rd, 2025.

• If your tenant was created before the API deprecation, you can use the current integration until March 2026.

• If your tenant was created since the API deprecation, you can use a PowerShell script to manully collect data using the new cmdlet, and use this integration to ingest it from a file. For more information, please see the "Logfile collection" section below.

• We're tracking Microsoft's new Graph-based Message Trace API (tentatively expected in December 2025) and plan to update this integration once it's available. Read Microsoft's announcement here.

The basic authentication configuration fields have been removed from this integration as Microsoft has deprecated and disabled basic authentication for Exchange Online. See the deprecation notification for details.

At a minimum, your Office 365 service account should include a role with Message Tracking and View‑Only Recipients permissions, assigned to the Office 365 user account that will be used for the integration. Assign these permissions using the Exchange admin center.

Logs are either gathered via the rest API or via a logfile. Log Documentation

The log dataset collects the Microsoft Exchange Online Message Trace logs. To search for ingested logs in Elasticsearch you need to query using datastream.dataset: microsoft_exchange_online_message_trace.log. This integration will poll the Microsoft Exchange Online Message Trace legacy API (https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace) to pull Message Trace logs and ingest them via the ingest pipelines.

In order to continue using the Microsoft Exchange Online Message Trace you will need to enable and configure OAuth2 authentication via your service app.

• Service App Configuration

  1. In the Azure portal, create a Microsoft Entra App (service app) Registration. For details please refer to the official Microsoft Documentation.
  2. In most cases under the Redirect URI section, you would want to configure the value Web for the app type and http://localhost for the Redirect URI, unless there are some specific requirements on your end.
  3. Assign the application at least one Microsoft Entra (Azure AD) role that will enable it to access the Reporting Web Service:
     • Security Reader
     • Global Reader
  4. The App Registration should contain the following API permissions: Office 365 Exchange Online > ReportingWebService.Read.All (application). See Specify the permissions your app requires to access the Reporting Web Service.

• Configuring OAuth2 Credentials

  Once you have your service app registered and configured, you can now configure your OAuth2 credentials as follows:-

  1. Generate a client secret for your registered service app. Copy and store the client secret value with you as this will be required for your OAuth2 credentials.

  2. Fill in the following fields with the appropriate values from your configured service app:-

     • Client ID: The client_id of your service app to pass in the OAuth request parameter.
     • Client secret: The client_secret of your service app that you generated earlier, to pass in the OAuth request parameter.
     • Tenant ID: The Directory ID (tenant identifier) of your service app in your Microsoft Entra ID(Azure Active Directory).

  With these values now configured, the OAuth2 configuration for the integration should be ideally complete. For more details, please check the official doc for Getting Started with Reporting Web Service.

• To configure Local Domains you can check your Microsoft Admin Exchange Center for the domains available in your organization. They are usually under the sections Accepted Domains and Remote Domains.

• The default Interval is configured to 1h and Initial Interval to 48h, you can however change these to your required values. The look-back value of Initial Interval should not exceed 200 hours as this might cause unexpected errors with the API.

• The default Minimum Age is configured to 1h, you can however change these to your required values. The Minimum Age was introduced to allow a sliding window to exist in combination with the Initial Interval. If you do not require a sliding window you can set this to 0s which will cause the Minimum Age to always default to the current time (now).

• The default Additional Look-back Time value is configured for 1h. This is intended to capture events that may not have been initially present due to eventual consistency. This value does not need to exceed 24h.

  • Note

    The larger this value is, the less likely events will be missed, however, this will cause the integration to take longer to pull all events, making newer events take longer to become present.

• The default value of Batch Size is set to 1000. This means for every request Httpjson will paginate with a value of 1000 results per page. The maximum page size supported by the Message Trace API is 2000. The API will return an empty value array when there are no more logs to pull and the pagination will terminate with an error that can be ignored.

Disclaimer: You may need to adapt the authentication method of the script below to match your environment. For more information about authentication methods available in PowerShell, please see the guides here. Note that basic authentication (with -Authentication Basic) is no longer supported.

The following example PowerShell script can be adapted to fetch the logs and write them into a JSON file that the integration can consume (via the logfile input).

Prerequisites:

Install the Exchange Online Management module by running the following command:

Install-Module -Name ExchangeOnlineManagement
		

Import the Exchange Online Management module by running the following command:

Import-Module -Name ExchangeOnlineManagement
		

This script would have to be triggered at a certain interval, in accordance with the look-back interval specified. In this example script, the look back is 24 hours, so the interval would need to be daily. For more information about the Get-MessageTraceV2 cmdlet, please refer to its documentation.

# Username and Password
$username = "USERNAME@DOMAIN.TLD"
$password = "PASSWORD"
# Lookback in Hours
$lookback = "-24"
# Results per request (maximum 5000)
$resultSize = "5000"
# Output of the json file
# This would then be ingested via the integration
$output_location = "C:\temp\messageTrace.json"

$password = ConvertTo-SecureString $password -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ($username, $password)
$startDate = (Get-Date).AddHours($lookback)
$endDate = Get-Date

Connect-ExchangeOnline -Credential $Credential

$paginate = 1
$output = @()

# Initialize V2-style pagination cursor values
$startingRecipientAddress = $null
$currentEndDate = $endDate

while ($paginate -eq 1)
{
    if ($startingRecipientAddress) {
        $messageTrace = Get-MessageTraceV2 -ResultSize $resultSize -StartDate $startDate -EndDate $currentEndDate -StartingRecipientAddress $startingRecipientAddress
    }
    else {
        $messageTrace = Get-MessageTraceV2 -ResultSize $resultSize -StartDate $startDate -EndDate $currentEndDate
    }

    if (!$messageTrace)
    {
        $paginate = 0
    }
    else
    {
        $output = $output + $messageTrace

        # If we got fewer than ResultSize rows, we've reached the end
        if ($messageTrace.Count -lt [int]$resultSize)
        {
            $paginate = 0
        }
        else
        {
            # Prepare the cursor data for the next query
            $last = $messageTrace[-1]
            $startingRecipientAddress = $last.RecipientAddress
            $currentEndDate = $last.Received
        }
    }
}

if (Test-Path $output_location)
{
    Remove-Item $output_location
}
foreach ($event in $output)
{
    $event.StartDate = [Xml.XmlConvert]::ToString(($event.StartDate), [Xml.XmlDateTimeSerializationMode]::Utc)
    $event.EndDate   = [Xml.XmlConvert]::ToString(($event.EndDate),   [Xml.XmlDateTimeSerializationMode]::Utc)
    $event.Received  = [Xml.XmlConvert]::ToString(($event.Received),  [Xml.XmlDateTimeSerializationMode]::Utc)
    $event = $event | ConvertTo-Json -Compress
    Add-Content $output_location $event -Encoding UTF8
}
		

{
    "@timestamp": "2022-10-21T17:25:36.969Z",
    "agent": {
        "ephemeral_id": "11edfb81-b112-45ba-8f01-6e7483e450fa",
        "id": "1c0788e9-492a-441e-acab-fc8c56281cf1",
        "name": "elastic-agent-22259",
        "type": "filebeat",
        "version": "8.19.4"
    },
    "data_stream": {
        "dataset": "microsoft_exchange_online_message_trace.log",
        "namespace": "71098",
        "type": "logs"
    },
    "destination": {
        "domain": "contoso.com",
        "registered_domain": "contoso.com",
        "top_level_domain": "com",
        "user": {
            "domain": "contoso.com",
            "email": "linus@contoso.com",
            "id": "linus@contoso.com",
            "name": "linus"
        }
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "1c0788e9-492a-441e-acab-fc8c56281cf1",
        "snapshot": false,
        "version": "8.19.4"
    },
    "email": {
        "attachments": {
            "file": {
                "size": 22761
            }
        },
        "delivery_timestamp": "2022-10-21T17:25:36.969376Z",
        "from": {
            "address": [
                "noreply@azure.microsoft.com"
            ]
        },
        "local_id": "a5e6dc0f-23df-4b20-d240-08dab38944a1",
        "message_id": "<GVAP278MB037586A65EF1FB2F844B0258DA2D9@GVAP278MB0375.CHEP278.PROD.OUTLOOK.COM>",
        "subject": "testmail 2",
        "to": {
            "address": [
                "linus@contoso.com"
            ]
        }
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "email"
        ],
        "dataset": "microsoft_exchange_online_message_trace.log",
        "ingested": "2025-10-06T13:13:06Z",
        "original": "{\"Organization\":\"contoso.com\",\"MessageId\":\"\\u003cGVAP278MB037586A65EF1FB2F844B0258DA2D9@GVAP278MB0375.CHEP278.PROD.OUTLOOK.COM\\u003e\",\"Received\":\"2022-10-21T17:25:36.969376Z\",\"SenderAddress\":\"noreply@azure.microsoft.com\",\"RecipientAddress\":\"linus@contoso.com\",\"Subject\":\"testmail 2\",\"Status\":\"Delivered\",\"ToIP\":null,\"FromIP\":\"40.107.23.54\",\"Size\":22761,\"MessageTraceId\":\"a5e6dc0f-23df-4b20-d240-08dab38944a1\",\"StartDate\":\"2022-10-21T09:40:10Z\",\"EndDate\":\"2022-10-22T09:40:10Z\",\"Index\":0}",
        "outcome": "success",
        "type": [
            "info"
        ]
    },
    "input": {
        "type": "log"
    },
    "log": {
        "file": {
            "path": "/tmp/service_logs/microsoft_exchange_online_message_trace_test.ndjson.log"
        },
        "offset": 0
    },
    "microsoft": {
        "online_message_trace": {
            "EndDate": "2022-10-22T09:40:10Z",
            "FromIP": "40.107.23.54",
            "Index": 0,
            "MessageId": "<GVAP278MB037586A65EF1FB2F844B0258DA2D9@GVAP278MB0375.CHEP278.PROD.OUTLOOK.COM>",
            "MessageTraceId": "a5e6dc0f-23df-4b20-d240-08dab38944a1",
            "Organization": "contoso.com",
            "Received": "2022-10-21T17:25:36.969376Z",
            "RecipientAddress": "linus@contoso.com",
            "SenderAddress": "noreply@azure.microsoft.com",
            "Size": 22761,
            "StartDate": "2022-10-21T09:40:10Z",
            "Status": "Delivered",
            "Subject": "testmail 2"
        }
    },
    "related": {
        "user": [
            "linus@contoso.com",
            "noreply@azure.microsoft.com",
            "linus",
            "noreply"
        ]
    },
    "source": {
        "domain": "azure.microsoft.com",
        "ip": "40.107.23.54",
        "registered_domain": "microsoft.com",
        "subdomain": "azure",
        "top_level_domain": "com",
        "user": {
            "domain": "azure.microsoft.com",
            "email": "noreply@azure.microsoft.com",
            "id": "noreply@azure.microsoft.com",
            "name": "noreply"
        }
    },
    "tags": [
        "preserve_original_event",
        "microsoft-defender-endpoint",
        "forwarded"
    ]
}
		

This integration includes one or more Kibana dashboards that visualizes the data collected by the integration. The screenshots below illustrate how the ingested data is displayed.

×