Microsoft 365

Collect logs from Microsoft 365 with Elastic Agent.

Version
2.4.0 (View all)
Compatible Kibana version(s)
8.13.0 or higher
Supported Serverless project types

Security
Observability
Subscription level
Basic
Level of support
Elastic

This integration is for Microsoft Office 365. It currently supports user, admin, system, and policy actions and events from Office 365 and Azure AD activity logs exposed by the Office 365 Management Activity API.

Setup

To use this package you need to enable Audit Log and register an application in Microsoft Entra ID (formerly known as Azure Active Directory).

Once the application is registered, configure and/or note the following to setup O365 Elastic integration:

  1. Note Application (client) ID and the Directory (tenant) ID in the registered application's Overview page.
  2. Create a new secret to configure the authentication of your application.
    • Navigate to Certificates & Secrets section.
    • Click New client secret and provide some description to create new secret.
    • Note the Value which is required for the integration setup.
  3. Add permissions to your registered application. Please check O365 Management API permissions for more details.
    • Navigate to API permissions page and click Add a permission
    • Select Office 365 Management APIs tile from the listed tiles.
    • Click Application permissions.
    • Under ActivityFeed, select ActivityFeed.Read permission. This is minimum required permissions to read audit logs of your organization as provided in the documentation. Optionally, select ActivityFeed.ReadDlp to read DLP policy events.
    • Click Add permissions.
    • If User.Read permission under Microsoft.Graph tile is not added by default, add this permission.
    • After the permissions are added, the admin has to grant consent for these permissions.

Once the secret is created and permissions are granted by admin, setup Elastic Agent's O365 integration:

  • Click Add Microsoft 365.
  • Enable Collect Office 365 audit logs via Management Activity API using CEL Input.
  • Add Directory (tenant) ID noted in Step 1 into Directory (tenant) ID parameter. This is required field.
  • Add Application (client) ID noted in Step 1 into Application (client) ID parameter. This is required field.
  • Add the secret Value noted in Step 2 into Client Secret parameter. This is required field.
  • Oauth2 Token URL can be added to generate the tokens during the oauth2 flow. If not provided, above Directory (tenant) ID will be used for oauth2 token generation.
  • Modify any other parameters as necessary.

NOTE: As Microsoft is no longer supporting Azure Active Directory Authentication Library (ADAL), the existing o365audit input is being deprecated in favor of new CEL input in version 1.18.0. Hence for versions >= 1.18.0, certificate based authentication (provided by earlier o365audit input) is no longer supported.

We request users upgrading from integration version < 1.18.0 to >= 1.18.0 to follow these steps:

  1. Upgrade the Elastic Stack version to >= 8.7.1.

  2. Upgrade the integration navigating via Integrations -> Microsoft 365 -> Settings -> Upgrade

  3. Upgrade the integration policy navigating via Integrations -> Microsoft 365 -> integration policies -> Version (Upgrade). If Upgrade option doesn't appear under the Version, that means the policy is already upgraded in the previous step. Please go to the next step.

  4. Modify the integration policy:

    • Disable existing configuration (marked as Deprecated) and enable Collect Office 365 audit logs via CEL configuration.
    • Add the required parameters such as Directory (tenant) ID, Application (client) ID, Client Secret based on the previous configuration.
    • Verify/Update Initial Interval configuration parameter to start fetching events from. This defaults to 7 days. Even if there is overlap in times, the events are not duplicated.
    • Update the other configuration parameters as required and hit Save Integration.

Please refer Upgrade an integration in case of any issues while performing integration upgrade.

Compatibility

The ingest-geoip and ingest-user_agent Elasticsearch plugins are required to run this module.

Logs

Audit

Uses the Office 365 Management Activity API to retrieve audit messages from Office 365 and Azure AD activity logs. These are the same logs that are available under Audit Log Search in the Security and Compliance Center.

An example event for audit looks as following:

{
    "@timestamp": "2020-02-07T16:43:53.000Z",
    "agent": {
        "ephemeral_id": "50dde7f7-f3a3-4597-9ce3-fd6c21fbe6df",
        "id": "a6ce2e4c-5271-405f-acc5-cb378534481d",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.12.1"
    },
    "client": {
        "address": "213.97.47.133",
        "ip": "213.97.47.133"
    },
    "data_stream": {
        "dataset": "o365.audit",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "a6ce2e4c-5271-405f-acc5-cb378534481d",
        "snapshot": false,
        "version": "8.12.1"
    },
    "event": {
        "action": "PageViewed",
        "agent_id_status": "verified",
        "category": [
            "web"
        ],
        "code": "SharePoint",
        "dataset": "o365.audit",
        "id": "99d005e6-a4c6-46fd-117c-08d7abeceab5",
        "ingested": "2024-04-01T12:10:04Z",
        "kind": "event",
        "original": "{Site=d5180cfc-3479-44d6-b410-8c985ac894e3, ObjectId=https://testsiem-my.sharepoint.com/personal/asr_testsiem_onmicrosoft_com/_layouts/15/onedrive.aspx, UserKey=i:0h.f|membership|1003200096971f55@live.com, ItemType=Page, OrganizationId=b86ab9d4-fcf1-4b11-8a06-7a8f91b47fbd, Operation=PageViewed, ClientIP=213.97.47.133, Workload=OneDrive, EventSource=SharePoint, RecordType=4, Version=1, UserId=asr@testsiem.onmicrosoft.com, WebId=8c5c94bb-8396-470c-87d7-8999f440cd30, CreationTime=2020-02-07T16:43:53, UserAgent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0, CustomUniqueId=true, Id=99d005e6-a4c6-46fd-117c-08d7abeceab5, CorrelationId=622b339f-4000-a000-f25f-92b3478c7a25, ListItemUniqueId=59a8433d-9bb8-cfef-6edc-4c0fc8b86875, UserType=0}",
        "outcome": "success",
        "provider": "OneDrive",
        "type": [
            "info"
        ]
    },
    "host": {
        "id": "b86ab9d4-fcf1-4b11-8a06-7a8f91b47fbd",
        "name": "testsiem.onmicrosoft.com"
    },
    "input": {
        "type": "cel"
    },
    "network": {
        "type": "ipv4"
    },
    "o365": {
        "audit": {
            "CorrelationId": "622b339f-4000-a000-f25f-92b3478c7a25",
            "CreationTime": "2020-02-07T16:43:53",
            "CustomUniqueId": true,
            "EventSource": "SharePoint",
            "ItemType": "Page",
            "ListItemUniqueId": "59a8433d-9bb8-cfef-6edc-4c0fc8b86875",
            "ObjectId": "https://testsiem-my.sharepoint.com/personal/asr_testsiem_onmicrosoft_com/_layouts/15/onedrive.aspx",
            "RecordType": "4",
            "Site": "d5180cfc-3479-44d6-b410-8c985ac894e3",
            "UserId": "asr@testsiem.onmicrosoft.com",
            "UserKey": "i:0h.f|membership|1003200096971f55@live.com",
            "UserType": "0",
            "Version": "1",
            "WebId": "8c5c94bb-8396-470c-87d7-8999f440cd30"
        }
    },
    "organization": {
        "id": "b86ab9d4-fcf1-4b11-8a06-7a8f91b47fbd"
    },
    "related": {
        "ip": [
            "213.97.47.133"
        ],
        "user": [
            "asr"
        ]
    },
    "source": {
        "ip": "213.97.47.133"
    },
    "tags": [
        "preserve_original_event",
        "forwarded",
        "o365-cel"
    ],
    "user": {
        "domain": "testsiem.onmicrosoft.com",
        "email": "asr@testsiem.onmicrosoft.com",
        "id": "asr@testsiem.onmicrosoft.com",
        "name": "asr"
    },
    "user_agent": {
        "device": {
            "name": "Mac"
        },
        "name": "Firefox",
        "original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0",
        "os": {
            "full": "Mac OS X 10.14",
            "name": "Mac OS X",
            "version": "10.14"
        },
        "version": "72.0."
    }
}

Exported fields

FieldDescriptionType
@timestamp
Event timestamp.
date
cloud.image.id
Image ID for the cloud instance.
keyword
data_stream.dataset
Data stream dataset name.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
host.containerized
If the host is a container.
boolean
host.os.build
OS build information.
keyword
host.os.codename
OS codename, if any.
keyword
input.type
Type of Filebeat input.
keyword
log.flags
Flags for the log file.
keyword
log.offset
Offset of the entry in the log file.
long
o365.audit.Activity
keyword
o365.audit.Actor.ID
keyword
o365.audit.Actor.Type
keyword
o365.audit.ActorContextId
keyword
o365.audit.ActorIpAddress
keyword
o365.audit.ActorUserId
keyword
o365.audit.ActorYammerUserId
keyword
o365.audit.AdditionalInfo.*
object
o365.audit.AlertEntityId
keyword
o365.audit.AlertId
keyword
o365.audit.AlertLinks
flattened
o365.audit.AlertType
keyword
o365.audit.AppAccessContext.*
object
o365.audit.AppId
keyword
o365.audit.ApplicationDisplayName
keyword
o365.audit.ApplicationId
keyword
o365.audit.AzureActiveDirectoryEventType
keyword
o365.audit.Category
keyword
o365.audit.ClientAppId
keyword
o365.audit.ClientIP
keyword
o365.audit.ClientIPAddress
keyword
o365.audit.ClientInfoString
keyword
o365.audit.ClientRequestId
keyword
o365.audit.Comments
text
o365.audit.CorrelationId
keyword
o365.audit.CreationTime
keyword
o365.audit.CustomUniqueId
boolean
o365.audit.Data.ad
keyword
o365.audit.Data.af
keyword
o365.audit.Data.aii
keyword
o365.audit.Data.ail
keyword
o365.audit.Data.alk
keyword
o365.audit.Data.als
keyword
o365.audit.Data.an
keyword
o365.audit.Data.at
date
o365.audit.Data.cid
keyword
o365.audit.Data.cpid
keyword
o365.audit.Data.dm
keyword
o365.audit.Data.dpn
keyword
o365.audit.Data.eid
keyword
o365.audit.Data.etps
keyword
o365.audit.Data.etype
keyword
o365.audit.Data.f3u
keyword
o365.audit.Data.flattened
The full Data document.
flattened
o365.audit.Data.fvs
keyword
o365.audit.Data.imsgid
keyword
o365.audit.Data.lon
keyword
o365.audit.Data.mat
keyword
o365.audit.Data.md
date
o365.audit.Data.ms
keyword
o365.audit.Data.od
keyword
o365.audit.Data.op
keyword
o365.audit.Data.ot
keyword
o365.audit.Data.plk
keyword
o365.audit.Data.pud
keyword
o365.audit.Data.reid
keyword
o365.audit.Data.rid
keyword
o365.audit.Data.sev
keyword
o365.audit.Data.sict
keyword
o365.audit.Data.sid
keyword
o365.audit.Data.sip
ip
o365.audit.Data.sitmi
keyword
o365.audit.Data.srt
keyword
o365.audit.Data.ssic
keyword
o365.audit.Data.suid
keyword
o365.audit.Data.tdc
keyword
o365.audit.Data.te
date
o365.audit.Data.thn
keyword
o365.audit.Data.tht
keyword
o365.audit.Data.tid
keyword
o365.audit.Data.tpid
keyword
o365.audit.Data.tpt
keyword
o365.audit.Data.trc
keyword
o365.audit.Data.ts
date
o365.audit.Data.tsd
keyword
o365.audit.Data.ttdt
date
o365.audit.Data.ttr
keyword
o365.audit.Data.upfc
keyword
o365.audit.Data.upfv
keyword
o365.audit.Data.ut
keyword
o365.audit.Data.von
keyword
o365.audit.Data.wl
keyword
o365.audit.Data.zfh
keyword
o365.audit.Data.zfn
keyword
o365.audit.Data.zmfh
keyword
o365.audit.Data.zmfn
keyword
o365.audit.Data.zu
keyword
o365.audit.DataType
keyword
o365.audit.EntityType
keyword
o365.audit.ErrorNumber
keyword
o365.audit.EventData
keyword
o365.audit.EventSource
keyword
o365.audit.ExceptionInfo.*
object
o365.audit.ExchangeMetaData.*
object
o365.audit.Experience
keyword
o365.audit.ExtendedProperties.*
object
o365.audit.ExternalAccess
boolean
o365.audit.FileSizeBytes
long
o365.audit.GroupName
keyword
o365.audit.Id
keyword
o365.audit.ImplicitShare
keyword
o365.audit.IncidentId
keyword
o365.audit.InterSystemsId
keyword
o365.audit.InternalLogonType
keyword
o365.audit.IntraSystemId
keyword
o365.audit.Item.*
object
o365.audit.Item.*.*
object
o365.audit.ItemName
keyword
o365.audit.ItemType
keyword
o365.audit.ListBaseType
keyword
o365.audit.ListId
keyword
o365.audit.ListItemUniqueId
keyword
o365.audit.LogonError
keyword
o365.audit.LogonType
keyword
o365.audit.LogonUserSid
keyword
o365.audit.MailboxGuid
keyword
o365.audit.MailboxOwnerMasterAccountSid
keyword
o365.audit.MailboxOwnerSid
keyword
o365.audit.MailboxOwnerUPN
keyword
o365.audit.Members
flattened
o365.audit.ModifiedProperties.*.*
object
o365.audit.Name
keyword
o365.audit.NewValue
keyword
o365.audit.ObjectDisplayName
keyword
o365.audit.ObjectId
keyword
o365.audit.ObjectType
keyword
o365.audit.Operation
keyword
o365.audit.OperationId
keyword
o365.audit.OperationProperties
object
o365.audit.OrganizationId
keyword
o365.audit.OrganizationName
keyword
o365.audit.OriginatingServer
keyword
o365.audit.Parameters.*
object
o365.audit.Platform
keyword
o365.audit.PolicyDetails
flattened
o365.audit.PolicyId
keyword
o365.audit.RecordType
keyword
o365.audit.RequestId
keyword
o365.audit.ResultStatus
keyword
o365.audit.SensitiveInfoDetectionIsIncluded
boolean
o365.audit.SessionId
keyword
o365.audit.Severity
keyword
o365.audit.SharePointMetaData.*
object
o365.audit.Site
keyword
o365.audit.SiteUrl
keyword
o365.audit.Source
keyword
o365.audit.SourceFileExtension
keyword
o365.audit.SourceFileName
keyword
o365.audit.SourceRelativeUrl
keyword
o365.audit.Status
keyword
o365.audit.SupportTicketId
keyword
o365.audit.Target.ID
keyword
o365.audit.Target.Type
keyword
o365.audit.TargetContextId
keyword
o365.audit.TargetUserOrGroupName
keyword
o365.audit.TargetUserOrGroupType
keyword
o365.audit.TeamGuid
keyword
o365.audit.TeamName
keyword
o365.audit.Timestamp
keyword
o365.audit.UniqueSharingId
keyword
o365.audit.UserAgent
keyword
o365.audit.UserId
keyword
o365.audit.UserKey
keyword
o365.audit.UserType
keyword
o365.audit.Version
keyword
o365.audit.WebId
keyword
o365.audit.Workload
keyword
o365.audit.WorkspaceId
keyword
o365.audit.WorkspaceName
keyword
o365.audit.YammerNetworkId
keyword

Changelog

VersionDetailsKibana version(s)

2.4.0

Enhancement View pull request
Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.

8.13.0 or higher

2.3.3

Bug fix View pull request
Improve handling of o365.audit.AdditionalInfo.

8.12.0 or higher

2.3.2

Bug fix View pull request
Improve handling of o365.audit.OperationProperties.

8.12.0 or higher

2.3.1

Bug fix View pull request
Check contentExpiration timestamp before requesting contents.

8.12.0 or higher

2.3.0

Enhancement View pull request
Set sensitive values as secret and fix incorrect mappings.

8.12.0 or higher

2.2.2

Bug fix View pull request
Fix dashboard panel filters

8.10.1 or higher

2.2.1

Bug fix View pull request
Prevent tenant ID being rendered into debug logs.

8.7.1 or higher

2.2.0

Enhancement View pull request
Expose OAuth2 token scopes configuration value.

8.7.1 or higher

2.1.2

Bug fix View pull request
Add error.message ECS field mapping.

8.7.1 or higher

2.1.1

Enhancement View pull request
Changed owners

8.7.1 or higher

2.1.0

Enhancement View pull request
Add extra values to related.user.

8.7.1 or higher

2.0.0

Enhancement View pull request
Mapping, parsing of o365.audit fields Platform and Data.*.

8.7.1 or higher

1.29.2

Enhancement View pull request
Add note to docs about configuring the Token URL.

8.7.1 or higher

1.29.1

Enhancement View pull request
Add required permissions to docs.

8.7.1 or higher

1.29.0

Enhancement View pull request
Limit request tracer log count to five.

8.7.1 or higher

1.28.0

Enhancement View pull request
ECS version updated to 8.11.0.

8.7.1 or higher

1.27.0

Enhancement View pull request
Allow user configuration of API request batch size.

8.7.1 or higher

1.26.0

Bug fix View pull request
Preserve 'event.original' from 'o365audit' field.

8.7.1 or higher

1.25.2

Bug fix View pull request
Prevent initial API call failure when a delayed call would result in a start time before 168h in the past.

8.7.1 or higher

1.25.1

Bug fix View pull request
Add start time fallback for responses that do not include the startTime query.

Bug fix View pull request
Add mappings for o365.audit.AdditionalInfo.* and o365.audit.AppAccessContext.*.

8.7.1 or higher

1.25.0

Enhancement View pull request
Improve 'event.original' check to avoid errors if set.

8.7.1 or higher

1.24.1

Enhancement View pull request
Fix mappings for dynamically mapped fields.

8.7.1 or higher

1.24.0

Enhancement View pull request
ECS version updated to 8.10.0.

8.7.1 or higher

1.23.0

Enhancement View pull request
The format_version in the package manifest changed from 2.11.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest.

8.7.1 or higher

1.22.0

Enhancement View pull request
Add tags.yml file so that integration's dashboards and saved searches are tagged with "Security Solution" and displayed in the Security Solution UI.

8.7.1 or higher

1.21.0

Enhancement View pull request
Increase CEL resource.tracer.maxsize to prevent loss of trace responses.

8.7.1 or higher

1.20.1

Bug fix View pull request
Fix timestamp error in CEL input

8.7.1 or higher

1.20.0

Enhancement View pull request
Update package to ECS 8.9.0.

8.7.1 or higher

1.19.0

Enhancement View pull request
Update package-spec to 2.9.0.

8.7.1 or higher

1.18.0

Enhancement View pull request
Migrate to CEL input from o365audit input

8.7.1 or higher

1.17.0

Enhancement View pull request
Convert visualizations to lens.

8.7.1 or higher

1.16.0

Enhancement View pull request
Ensure event.kind is correctly set for pipeline errors.

8.1.0 or higher

1.15.0

Enhancement View pull request
Update package to ECS 8.8.0.

8.1.0 or higher

1.14.1

Enhancement View pull request
Add FileSizeBytes field in o365.audit

8.1.0 or higher

1.14.0

Enhancement View pull request
Update package to ECS 8.7.0.

8.1.0 or higher

1.13.1

Enhancement View pull request
Added categories and/or subcategories.

8.1.0 or higher

1.13.0

Enhancement View pull request
Add NewValue field to o365.audit.

8.1.0 or higher

1.12.0

Enhancement View pull request
Add NewValue field to o365.audit.

8.1.0 or higher

1.11.0

Enhancement View pull request
Retain file, directory and URL details for SharePointSharingOperation.

8.1.0 or higher

1.10.1

Enhancement View pull request
Add fingerprint processor to prevent ingestion of duplicate events.

8.1.0 or higher

1.10.0

Enhancement View pull request
Update package to ECS 8.6.0.

8.1.0 or higher

1.9.2

Enhancement View pull request
Migrate the visualizations to by value in dashboards to minimize the saved object clutter and reduce time to load

8.1.0 or higher

1.9.1

Bug fix View pull request
Remove duplicate fields.

7.14.0 or higher
8.0.0 or higher

1.9.0

Enhancement View pull request
Update package to ECS 8.5.0.

7.14.0 or higher
8.0.0 or higher

1.8.2

Enhancement View pull request
Use ECS geo.location definition.

7.14.0 or higher
8.0.0 or higher

1.8.1

Enhancement View pull request
Added link to Microsoft documentation.

7.14.0 or higher
8.0.0 or higher

1.8.0

Enhancement View pull request
Update package to ECS 8.4.0

7.14.0 or higher
8.0.0 or higher

1.7.1

Enhancement View pull request
Update package name and description to align with standard wording

7.14.0 or higher
8.0.0 or higher

1.7.0

Enhancement View pull request
Update package to ECS 8.3.0.

7.14.0 or higher
8.0.0 or higher

1.6.0

Enhancement View pull request
Extend list of mapped record types

7.14.0 or higher
8.0.0 or higher

1.5.1

Bug fix View pull request
Fix processing of ModifiedProperties when it is a list of strings

7.14.0 or higher
8.0.0 or higher

1.5.0

Enhancement View pull request
Update to ECS 8.2

1.4.3

Enhancement View pull request
Replace invalid field value

7.14.0 or higher
8.0.0 or higher

1.4.2

Enhancement View pull request
Add documentation for multi-fields

7.14.0 or higher
8.0.0 or higher

1.4.1

Bug fix View pull request
Fix grok parsing errors due to invalid IP addresses.

7.14.0 or higher
8.0.0 or higher

1.4.0

Enhancement View pull request
Update to ECS 8.0

7.14.0 or higher
8.0.0 or higher

1.3.2

Bug fix View pull request
Regenerate test files using the new GeoIP database

7.14.0 or higher
8.0.0 or higher

1.3.1

Bug fix View pull request
Change test public IPs to the supported subset

1.3.0

Enhancement View pull request
Add 8.0.0 version constraint

7.14.0 or higher
8.0.0 or higher

1.2.4

Enhancement View pull request
Uniform with guidelines

7.14.0 or higher

1.2.3

Bug fix View pull request
Fix processors configuration

7.14.0 or higher

1.2.2

Enhancement View pull request
Update Title and Description.

1.2.1

Bug fix View pull request
Fix logic that checks for the 'forwarded' tag

1.2.0

Enhancement View pull request
Update to ECS 1.12.0

1.1.4

Bug fix View pull request
additional type assertions in painless scripts

7.14.0 or higher

1.1.3

Enhancement View pull request
Convert to generated ECS fields

1.1.2

Enhancement View pull request
update to ECS 1.11.0

1.1.1

Enhancement View pull request
Escape special characters in docs

1.1.0

Enhancement View pull request
Update integration description

1.0.0

Enhancement View pull request
make GA

Enhancement View pull request
Set "event.module" and "event.dataset"

7.14.0 or higher

0.6.2

Enhancement View pull request
Use wildcard field type for the relevant ECS fields.

0.6.1

Enhancement View pull request
small pipeline fix for parameters when they are string instead of objects

0.6.0

Enhancement View pull request
add system tests

Enhancement View pull request
fixes o365audit config file

0.5.0

Enhancement View pull request
update to ECS 1.10.0 and adding event.original options

0.4.0

Enhancement View pull request
moving edge processing to ingest pipelines

0.3.2

Enhancement View pull request
update to ECS 1.9.0

0.3.1

Bug fix View pull request
Change kibana.version constraint to be more conservative.

0.3.0

Enhancement View pull request
Add changes to use ECS 1.8 fields.

0.1.0

Enhancement View pull request
initial release

On this page