Snowflake Integration for Elastic

Version 0.2.0 (View all)
Subscription level
What's this?
Basic
Developed by
What's this?
Elastic
Ingestion method(s) API
Minimum Kibana version(s) 9.1.0
8.19.0
The Snowflake integration v0.2.0 is in technical preview

To use pre-release integrations, go to the Integrations page in Kibana, scroll down, and toggle on the Display beta integrations option.

The Snowflake integration for Elastic collects security telemetry from Snowflake ACCOUNT_USAGE views using the SQL API. It covers object and column access, cross-cloud and cross-region data transfer, and stage lifecycle changes so you can monitor data access and possible exfiltration paths in Elasticsearch.

This integration queries Snowflake ACCOUNT_USAGE historical views through SQL API v2 (/api/v2/statements). Those views retain about one year of data and are delayed (about 45 minutes to 3 hours depending on the view). It is compatible with Snowflake accounts on AWS, Azure, and Google Cloud, including PrivateLink hosts. China region hosts (.snowflakecomputing.cn) are supported when the account URL is set accordingly. The access_history data stream requires Snowflake Enterprise Edition or later.

Elastic Agent submits SQL statements to POST /api/v2/statements and pages result partitions. If Snowflake returns HTTP 202, the agent polls the statement handle until results are ready. Each data stream queries one ACCOUNT_USAGE view and advances a timestamp watermark with lookback to cover view latency. When a statement returns a full batch_size page, the next request continues from the last returned ORDER BY key without the lag overlap until the backlog is drained. Grants keep separate watermarks for CREATED_ON and DELETED_ON.

The Snowflake integration collects the following logs:

  • access_history: Objects and columns read or written by each query from SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY (direct and base objects accessed, objects modified, DDL operations, policies referenced). Enterprise Edition or later.
  • data_transfer_history: Hourly byte-count windows of data moved between clouds and regions from SNOWFLAKE.ACCOUNT_USAGE.DATA_TRANSFER_HISTORY, by transfer type.
  • stages: Stage create, alter, and drop activity from SNOWFLAKE.ACCOUNT_USAGE.STAGES, including external stage locations (s3://, azure://, gcs://) and storage integrations. Watermarks are LAST_ALTERED and DELETED because rows update in place.
  • login_history: Login attempts from SNOWFLAKE.ACCOUNT_USAGE.LOGIN_HISTORY (user, client IP, authentication factors, success or failure, malicious-IP protection details).
  • sessions: Session lifecycle from SNOWFLAKE.ACCOUNT_USAGE.SESSIONS (open or closed sessions, last access time, closed reason). Watermark is ACCESS_TIME because session rows update until they close.
  • grants_to_users: Role grants and revokes from SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_USERS. Creates and revokes are queried separately so each watermark advances only along CREATED_ON or DELETED_ON.

Use this integration to detect failed or suspicious logins, track session activity, audit privilege grants and revokes, monitor access to sensitive tables and columns, detect unusual outbound data transfers, and track the creation or alteration of external stages that could facilitate data exfiltration from Snowflake.

  • An Elastic Stack deployment (self-managed or Elastic Cloud) that meets the package Kibana version condition.
  • A Snowflake account and a virtual warehouse that can auto-resume.
  • A Snowflake TYPE=SERVICE user with a role that can query the views and use the warehouse.
  • A programmatic access token for the service user. Username/password, OAuth access tokens, and key-pair JWT are not supported by this integration.

Elastic Agent must be installed. For more details, check the Elastic Agent installation instructions. You can install only one Elastic Agent per host.

Elastic Agent is required to stream data from the SQL API and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines.

  1. Create a role and a TYPE=SERVICE user, and grant the role to the user.
  2. Grant USAGE on a warehouse that can auto-resume.
  3. Grant the SNOWFLAKE.SECURITY_VIEWER database role (or IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE) so the user can read login history, sessions, and grants to users.
  4. Grant the SNOWFLAKE database roles the data streams need (or IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE):
    • SNOWFLAKE.GOVERNANCE_VIEWER for access_history
    • SNOWFLAKE.USAGE_VIEWER for data_transfer_history
    • SNOWFLAKE.OBJECT_VIEWER for stages
    • SNOWFLAKE.SECURITY_VIEWER database role or IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE for login history, sessions, and grants_to_users
  5. Create a programmatic access token for the service user.
  6. If a network policy is enabled, allow the Elastic Agent egress IP (or PrivateLink endpoint).
  1. In Kibana go to Management → Integrations.
  2. Search for Snowflake and add the integration.
  3. Set the account URL, role, and warehouse.
  4. Provide the programmatic access token for the service user. OAuth access tokens and key-pair JWT are not supported.
  5. Enable the data streams you want and save the policy.

In Discover, filter on event.dataset: snowflake.access_history, event.dataset: snowflake.data_transfer_history, event.dataset: snowflake.stages, event.dataset: snowflake.login_history, event.dataset: snowflake.sessions, or event.dataset: snowflake.grants_to_users. Access history should contain event.category: database events. Data transfer should contain event.category: network events with network.bytes. Stages should contain event.category: configuration events with event.action of create, alter, or drop. Login history should contain event.category: authentication events. Grants should contain IAM change events.

  • No data is being collected: Confirm the warehouse can auto-resume, the role has the required SNOWFLAKE database role (or imported privileges), and the account URL is reachable on HTTPS 443. ACCOUNT_USAGE latency means new rows can take up to 2–3 hours to appear.
  • access_history returns an error: The ACCESS_HISTORY view exists only on Enterprise Edition or later. Turn off the data stream on lower editions.
  • Authentication failures: Service users cannot use passwords. Provide a programmatic access token. The integration always sends X-Snowflake-Authorization-Token-Type: PROGRAMMATIC_ACCESS_TOKEN.
  • Network policy blocks the SQL API: Add the agent egress IP, or use a PrivateLink hostname from SYSTEM$GET_PRIVATELINK_CONFIG().
  • HTTP 429 (390505): Reduce poll frequency. There is no documented Retry-After contract on the SQL API.
  • Duplicate rows: Each poll overlaps the previous watermark by the configured lag time so late-arriving rows are included. Documents use a fingerprint _id, so re-delivered rows are usually dropped as duplicates, never overwritten, and a few duplicates can appear across index rollovers.

Each poll runs SQL on the configured warehouse and consumes Snowflake credits. ACCOUNT_USAGE latency is 45 minutes to 3 hours, so polling faster than the view latency does not make data fresher. Suggested starting intervals are 2h for login history, grants, data transfer history, and stages, and 3h for sessions and access history. Overlap the watermark by at least the view latency so late-arriving rows are included. ACCESS_HISTORY can be high volume on busy accounts. Keep batch_size at the default unless statements time out.

For more information on architectures that can be used for scaling this integration, check the Ingest Architectures documentation.

These inputs can be used with this integration:

To collect logs via API endpoint, configure the following parameters:

  • API Endpoint URL
  • API credentials (tokens, keys, or username/password)
  • Request interval (how often to fetch data)

These APIs are used with this integration:

The access_history data stream provides object and column access events from SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY. Nested object lists are decoded from the view's JSON columns and kept as flattened fields. Object names and column names (which vary per query) are also extracted into keyword arrays for aggregation.

The data_transfer_history data stream provides hourly data transfer windows from SNOWFLAKE.ACCOUNT_USAGE.DATA_TRANSFER_HISTORY. Bytes transferred are mapped to network.bytes and the transfer type to event.action.

The stages data stream provides stage lifecycle events from SNOWFLAKE.ACCOUNT_USAGE.STAGES. Rows update in place, so the collector watermarks on LAST_ALTERED and DELETED, and the pipeline derives create, alter, or drop from the timestamps.

The login_history data stream provides login attempt events from SNOWFLAKE.ACCOUNT_USAGE.LOGIN_HISTORY.

The sessions data stream provides session lifecycle events from SNOWFLAKE.ACCOUNT_USAGE.SESSIONS. Rows update until the session closes. The collector watermarks on ACCESS_TIME.

The grants_to_users data stream provides role grant and revoke events from SNOWFLAKE.ACCOUNT_USAGE.GRANTS_TO_USERS. Creates and revokes are collected with separate statements so a revoke timestamp cannot skip unread grants.

This integration includes one or more Kibana dashboards that visualizes the data collected by the integration. The screenshots below illustrate how the ingested data is displayed.