Privileged Access Detection

The Privileged Access Detection package contains assets to detect anomalous privilege access activity in the Windows, Linux and Okta system logs. This package requires a Platinum subscription. Please ensure that you have a Trial, Platinum, or Enterprise subscription before proceeding. This package is licensed under Elastic License 2.0.

The package transform supports data from Elastic Endpoint via Elastic Defend and the Okta integration. Prior to using this integration, either the Okta integration should be installed and configured, and/or Elastic Defend should be installed through Elastic Agent and collecting data from hosts, or have equivalent tools/endpoints set up. See Configure endpoint protection with Elastic Defend and Okta Integration for more information. The Transform and Anomaly Detection Jobs sections have detailed platform support.

1. Upgrading: If upgrading from a version below v2.0.0, see the section v2.0.0 and beyond.

2. Add the Integration Package: Install the package via Management > Integrations > Add Privileged Access Detection. Configure the integration name and agent policy. Click Save and Continue.

3. Configure the pipeline: To configure the pipeline you can use one of the following steps:

   • If using Elastic Defend, add a custom pipeline to the data stream. Go to Stack Management > Ingest Pipelines, and check if the pipeline logs-endpoint.events.process@custom exists. If it does not exist, you can create it by running the following command in the Dev Console. Be sure to replace <VERSION> with the current package version.

     PUT _ingest/pipeline/logs-endpoint.events.process@custom
     {
       "processors": [
         {
           "pipeline": {
             "name": "<VERSION>-ml_pad_ingest_pipeline",
             "ignore_missing_pipeline": true,
             "ignore_failure": true
           }
         }
       ]
     }
     		

   • If logs-endpoint.events.process@custom already exists, select the three dots next to it and choose Edit. Click Add a processor. Select Pipeline for Processor, enter <VERSION>-ml_pad_ingest_pipeline for name (replacing <VERSION> with the current package version), and check Ignore missing pipeline and Ignore failures for this processor. Select Add Processor.

4. Add the required mappings to the component template: Go to Stack Management > Index Management > Component Templates. Templates that can be edited to add custom components will be marked with a @custom suffix. For instance, the custom component template for Elastic Defend process events is logs-endpoint.events.process@custom. Note: Do not attempt to edit the @package template.

   • If the @custom component template does not exist, you can execute the following command in the Dev Console to create it and then continue to the Rollover section in these instructions. Be sure to change <VERSION> to the current package version.

     PUT _component_template/{COMPONENT_TEMPLATE_NAME}@custom
     {
       "template": {
         "mappings": {
           "properties": {
             "process": {
               "type": "object",
               "properties": {
                 "command_line_entropy": {
                   "type": "double"
                 }
               }
             }
           }
         }
       }
     }
     		

   • If the @custom component template already exists, you will need to edit it to add mappings for data to be properly enriched. Click the three dots next to it and select Edit.
   • Proceed to the mappings step in the UI. Click Add Field at the bottom of the page and create an Object field for process:
   • Create a property under Process for command_line_entropy of type Double.
   • Your component mappings should look like the following:
   • Click Review then Save Component Template.

5. Rollover Depending on your environment, you may need to rollover in order for these mappings to get picked up. The default index pattern for Elastic Defend is logs-endpoint.events.process-default.

   POST INDEX_NAME/_rollover
   		

6. Check the health of the transforms: The transforms are scheduled to run every hour. These transforms create two indices: ml_windows_privilege_type_pad_ea.all and ml_okta_multiple_user_sessions_pad_ea.all. To check the health of the transforms go to Management > Stack Management > Data > Transforms under logs-pad.pivot_transform_okta_sessions_ea-default-<FLEET-TRANSFORM-VERSION> and logs-pad.pivot_transform_win_privilege_list_ea-default-<FLEET-TRANSFORM-VERSION>.

7. Create a data view for anomaly detection jobs: The anomaly detection jobs under this package rely on three indices. One index contains logs for Windows, Linux, and Okta (logs-*), while the second and third indices store Okta user session information and details about special Windows privileges assigned to a user, respectively, collected through two transforms (ml_okta_multiple_user_sessions_pad_ea.all and ml_windows_privilege_type_pad_ea.all). The Okta session and Windows privilege jobs use their respective designated indices populated by the transforms installed above, which are already pre-assigned to those jobs. Before enabling the anomaly detection jobs, create a data view for the logs-* (or a similar index pattern) that includes data for Windows, Linux, and Okta events.

8. Add preconfigured anomaly detection jobs: In Stack Management -> Anomaly Detection Jobs, you will see Select data view or saved search. Select the data view created in the previous step. Then under Use preconfigured jobs you will see Privileged Access Detection. When you select the card, you will see pre-configured anomaly detection jobs that you can create depending on what makes the most sense for your environment. Note: In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the pad-ml file. Additionally, we recommend backdating the datafeed for these anomaly detection jobs to a specific timeframe, as some datafeed queries are resource-intensive and may lead to query delays. We advise you to start the datafeed with 2-3 months' worth of data.

9. Data view configuration for Dashboards: For the dashboard to work as expected, the following settings need to be configured in Kibana.

   1. You have started the above anomaly detection jobs.
   2. You have read access to .ml-anomalies-shared data stream/index or are assigned the machine_learning_user role. For more information on roles, please refer to Built-in roles in Elastic. Please be aware that a user who has access to the underlying machine learning results indices can see the results of all jobs in all spaces. Be mindful of granting permissions if you use Kibana spaces to control which users can see which machine learning results. For more information on machine learning privileges, refer to setup-privileges.
   3. After enabling the jobs, go to Management > Stack Management > Kibana > Data Views. Click on Create data view with the following settings:
      • Name: .ml-anomalies-shared
      • Index pattern : .ml-anomalies-shared*
      • Select Show Advanced settings enable Allow hidden and system indices
      • Custom data view ID: .ml-anomalies-shared

   Warning: When creating the data views for the dashboards, ensure that the Custom data view ID is set to the value specified above and is not left empty. Omitting or misconfiguring this field may result in broken visualizations, as illustrated by the error message below.

10. Enabling detection rules: You can also enable detection rules to alert on Privileged Access activity in your environment, based on anomalies flagged by the above ML jobs. These rules are available as part of the Detection Engine, and can be found using the tag Use Case: Privileged Access Detection. See this documentation for more information on importing and enabling the rules.

To inspect the installed assets, you can navigate to Stack Management > Data > Transforms.

When querying the destination indices for Okta and Windows logs, we advise using the alias for the destination index (ml_okta_multiple_user_sessions_pad_ea.all and ml_windows_privilege_type_pad_ea.all). In the event that the underlying package is upgraded, the alias will aid in maintaining the previous findings.

To customize filters in the Privileged Access Detection transform, follow the below steps. You can use these instructions to update basic settings or to update filters for fields such as process.name, @timestamp and others.

1. To update settings such as retention policy, frequency, or destination configuration, stop the transform, click Edit from the Actions bar, make the required changes, and start the transform again.
2. To update the query filters, go to Stack Management > Data > Transforms > logs-pad.pivot_transform_win_privilege_list_ea-default-<FLEET-TRANSFORM-VERSION>.
3. Click on the Actions bar at the far right of the transform and select the Clone option.
4. In the new Clone transform window, go to the Search filter and update any field values you want to add or remove. Click on the Apply changes button on the right side to save these changes. Note: The image below shows an example of filtering a new process.name as explorer.exe. You can follow a similar example and update the field value list based on your environment to help reduce noise and potential false positives.
5. Scroll down and select the Next button at the bottom right. Under the Transform details section, enter a new Transform ID and Destination index of your choice, then click on the Next button.
6. Lastly, select the Create and Start option. Your updated transform will now start collecting data. Note: Do not forget to update your data view based on the new Destination index you have just created.

To customize the datafeed query and other settings such as model memory limit, frequency, query delay, bucket span and influencers for the Privileged Access Detection ML jobs, follow the steps below.

1. To update the datafeed query, stop the datafeed and select Edit job from the Actions menu.
2. In the Edit job window, navigate to the Datafeed section and update the query filters. You can add or remove field values to help reduce noise and false positives based on your environment.
3. You may also update the model memory limit if your environment has high data volume or if the job requires additional resources. Go to the Job details section and update the Model memory limit and hit Save. For more information on resizing ML jobs, refer to the documentation.
4. In order to do more advanced changes to your job, clone the job by selecting Clone job from the Actions menu.
5. In the cloned job, you can update datafeed settings such as Frequency and Query delay, which help control how often data is analyzed and account for ingestion delays.
6. You can also modify the job configuration by adjusting the Bucket span and by adding or removing Influencers to improve anomaly attribution.
7. Finally, assign a new Job ID, and click on Create job, and start the datafeed to apply the updated settings.

v2.0.0 of this package requires Elastic Stack version 9.4 or later. It introduces support for Entity Analytics (EA), adding new fields for proper entity resolution.

• This package installs new ML jobs which include _ea suffix in their names, as outlined below. New transforms and detection rules are also included.
• Previously installed ML jobs, transforms, and rules will continue to run, allowing time to transition to the new Entity Analytics assets.
• Important: We recommend installing the new ML jobs and transforms and verifying that they are properly set up, collecting data, and generating anomalies before deleting the old jobs and upgrading to the new version of the detection rules available in 9.4. The new detection rules reference ML job IDs with the _ea suffix and are not compatible with older versions of the jobs.
• The new Entity Analytics transforms write to separate destination indices postfixed with _ea. Create a new data view for the Entity Analytics anomaly detection jobs using the new destination indices/aliases listed below. Do not mix old and new transform destination indices in the same data view.
• New dashboards are available in this version with the suffix "(Entity Analytics)" in the title. If you are still running jobs or transforms from before this version, the original dashboards without the suffix remain available.

The new Entity Analytics ML job IDs are:

• pad_windows_high_count_special_logon_events_ea
• pad_windows_high_count_special_privilege_use_events_ea
• pad_windows_high_count_group_management_events_ea
• pad_windows_high_count_user_account_management_events_ea
• pad_windows_rare_privilege_assigned_to_user_ea
• pad_windows_rare_group_name_by_user_ea
• pad_windows_rare_device_by_user_ea
• pad_windows_rare_source_ip_by_user_ea
• pad_windows_rare_region_name_by_user_ea
• pad_linux_high_count_privileged_process_events_by_user_ea
• pad_linux_rare_process_executed_by_user_ea
• pad_linux_high_median_process_command_line_entropy_by_user_ea
• pad_okta_spike_in_group_membership_changes_ea
• pad_okta_spike_in_user_lifecycle_management_changes_ea
• pad_okta_spike_in_group_privilege_changes_ea
• pad_okta_spike_in_group_application_assignment_changes_ea
• pad_okta_spike_in_group_lifecycle_changes_ea
• pad_okta_high_sum_concurrent_sessions_by_user_ea
• pad_okta_rare_source_ip_by_user_ea
• pad_okta_rare_region_name_by_user_ea
• pad_okta_rare_host_name_by_user_ea

The new Entity Analytics transforms are:

• pad.pivot_transform_okta_sessions_ea → destination index: ml_okta_multiple_user_sessions_pad_ea-2.0.0, alias: ml_okta_multiple_user_sessions_pad_ea.latest, ml_okta_multiple_user_sessions_pad_ea.all
• pad.pivot_transform_win_privilege_list_ea → destination index: ml_windows_privilege_type_pad_ea-2.0.0, alias: ml_windows_privilege_type_pad_ea.latest, ml_windows_privilege_type_pad_ea.all

After confirming the new Entity Analytics ML jobs and transforms are running correctly, you can remove the following deprecated assets that have been superseded by the new Entity Analytics versions (Elastic stack 9.4+):

• Delete old ML jobs: Navigate to Stack Management -> Anomaly Detection Jobs and delete the following jobs:
  • pad_windows_high_count_special_logon_events
  • pad_windows_high_count_special_privilege_use_events
  • pad_windows_high_count_group_management_events
  • pad_windows_high_count_user_account_management_events
  • pad_windows_rare_privilege_assigned_to_user
  • pad_windows_rare_group_name_by_user
  • pad_windows_rare_device_by_user
  • pad_windows_rare_source_ip_by_user
  • pad_windows_rare_region_name_by_user
  • pad_linux_high_count_privileged_process_events_by_user
  • pad_linux_rare_process_executed_by_user
  • pad_linux_high_median_process_command_line_entropy_by_user
  • pad_okta_spike_in_group_membership_changes
  • pad_okta_spike_in_user_lifecycle_management_changes
  • pad_okta_spike_in_group_privilege_changes
  • pad_okta_spike_in_group_application_assignment_changes
  • pad_okta_spike_in_group_lifecycle_changes
  • pad_okta_high_sum_concurrent_sessions_by_user
  • pad_okta_rare_source_ip_by_user
  • pad_okta_rare_region_name_by_user
  • pad_okta_rare_host_name_by_user
• Delete old transforms: Navigate to Stack Management -> Data -> Transforms and delete:
  • pad.pivot_transform_okta_multiple_sessions
  • pad.pivot_transform_windows_privilege_list

Usage in production requires that you have a license key that permits use of machine learning features.

This integration includes one or more Kibana dashboards that visualizes the data collected by the integration. The screenshots below illustrate how the ingested data is displayed.

×