STAN

Collect logs and metrics from STAN servers with Elastic Agent.

What is an Elastic integration?
Version
1.7.0 (View all)
Compatible Kibana version(s)
8.13.0 or higher
Supported Serverless project types

What's this?
Security
Observability
Subscription level
What's this?
Basic

This integration is used to collect logs and metrics from STAN servers. The integration collects metrics from STAN monitoring server APIs.

Compatibility

The STAN package is tested with Stan 0.15.1.

Logs

log

The log dataset collects the STAN logs.

An example event for log looks as following:

{
    "@timestamp": "2024-06-18T10:41:22.515Z",
    "agent": {
        "ephemeral_id": "e925a178-f704-43e6-899e-b69b164425d8",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.log",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "created": "2024-06-18T10:41:36.382Z",
        "dataset": "stan.log",
        "ingested": "2024-06-18T10:41:38Z",
        "kind": "event",
        "type": [
            "info"
        ]
    },
    "input": {
        "type": "log"
    },
    "log": {
        "file": {
            "path": "/tmp/service_logs/stan.log"
        },
        "level": "info",
        "offset": 0
    },
    "message": "STREAM: Starting nats-streaming-server[test-cluster] version 0.15.1",
    "process": {
        "pid": 7
    },
    "stan": {
        "log": {
            "msg": {}
        }
    },
    "tags": [
        "forwarded",
        "stan-log"
    ]
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields

FieldDescriptionType
@timestamp
Event timestamp.
date
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
input.type
Type of Filebeat input.
keyword
log.offset
Offset of the entry in the log file.
long
stan.log.client.id
The id of the client
integer
stan.log.msg.bytes
Size of the payload in bytes
long
stan.log.msg.error.message
Details about the error occurred
text
stan.log.msg.max_messages
An optional number of messages to wait for before automatically unsubscribing
integer
stan.log.msg.queue_group
The queue group which subscriber will join
text
stan.log.msg.reply_to
The inbox subject on which the publisher is listening for responses
keyword
stan.log.msg.sid
The unique alphanumeric subscription ID of the subject
integer
stan.log.msg.subject
Subject name this message was received on
keyword
stan.log.msg.type
The protocol message type
keyword

Metrics

The default datasets are stats, channels, and subscriptions.

stats

This is the stats dataset of the STAN package, in charge of retrieving generic metrics from a STAN instance.

An example event for stats looks as following:

{
    "@timestamp": "2024-06-18T10:42:35.470Z",
    "agent": {
        "ephemeral_id": "b30d1b07-36f0-4e5f-9f80-56b6015fb855",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.stats",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.stats",
        "duration": 1823495,
        "ingested": "2024-06-18T10:42:47Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "8259e024976a406e8a54cdbffeb84fec",
        "ip": "192.168.252.5",
        "mac": "02-42-C0-A8-FC-05",
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "3.10.0-1160.102.1.el7.x86_64",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.6 LTS (Focal Fossa)"
        }
    },
    "metricset": {
        "name": "stats",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/serverz",
        "type": "stan"
    },
    "stan": {
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "I9vNI3muOuNoem5vuoyo5z"
        },
        "stats": {
            "bytes": 0,
            "channels": 1,
            "clients": 100,
            "messages": 0,
            "state": "STANDALONE",
            "subscriptions": 100
        }
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields

FieldDescriptionTypeMetric Type
@timestamp
Event timestamp.
date
agent.id
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.
keyword
cloud.account.id
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.
keyword
cloud.availability_zone
Availability zone in which this host, resource, or service is located.
keyword
cloud.instance.id
Instance ID of the host machine.
keyword
cloud.provider
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
keyword
cloud.region
Region in which this host, resource, or service is located.
keyword
container.id
Unique container id.
keyword
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
host.name
Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.
keyword
service.address
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).
keyword
stan.cluster.id
The cluster ID
keyword
stan.server.id
The server ID
keyword
stan.stats.bytes
Number of bytes consumed across all STAN queues
long
counter
stan.stats.channels
The number of STAN channels
integer
gauge
stan.stats.clients
The number of STAN clients
integer
gauge
stan.stats.messages
Number of messages across all STAN queues
long
counter
stan.stats.role
If clustered, role of this node in the cluster (Leader, Follower, Candidate)
keyword
stan.stats.state
The cluster / streaming configuration state (STANDALONE, CLUSTERED)
keyword
stan.stats.subscriptions
The number of STAN streaming subscriptions
integer
gauge

channels

This is the channels dataset of the STAN package, in charge of retrieving metrics about channels from a STAN instance.

An example event for channels looks as following:

{
    "@timestamp": "2024-06-18T10:40:33.161Z",
    "agent": {
        "ephemeral_id": "2f2befc0-8f03-49d7-b5bf-8131709857b6",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.channels",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.channels",
        "duration": 3556501,
        "ingested": "2024-06-18T10:40:45Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "8259e024976a406e8a54cdbffeb84fec",
        "ip": "192.168.252.5",
        "mac": "02-42-C0-A8-FC-05",
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "3.10.0-1160.102.1.el7.x86_64",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.6 LTS (Focal Fossa)"
        }
    },
    "metricset": {
        "name": "channels",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
        "type": "stan"
    },
    "stan": {
        "channels": {
            "bytes": 0,
            "depth": 0,
            "first_seq": 0,
            "last_seq": 0,
            "messages": 0,
            "name": "foo"
        },
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "KJXiShxwxsi2oNAUH1Rlzn"
        }
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields

FieldDescriptionTypeMetric Type
@timestamp
Event timestamp.
date
agent.id
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.
keyword
cloud.account.id
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.
keyword
cloud.availability_zone
Availability zone in which this host, resource, or service is located.
keyword
cloud.instance.id
Instance ID of the host machine.
keyword
cloud.provider
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
keyword
cloud.region
Region in which this host, resource, or service is located.
keyword
container.id
Unique container id.
keyword
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
host.name
Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.
keyword
service.address
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).
keyword
stan.channels.bytes
The number of STAN bytes in the channel
long
counter
stan.channels.depth
Queue depth based upon current sequence number and highest reported subscriber sequence number
long
gauge
stan.channels.first_seq
First sequence number stored in the channel. If first_seq > min([seq in subscriptions]) data loss has possibly occurred
long
stan.channels.last_seq
Last sequence number stored in the channel
long
counter
stan.channels.messages
The number of STAN streaming messages
long
counter
stan.channels.name
The name of the STAN streaming channel
keyword
stan.cluster.id
The cluster ID
keyword
stan.server.id
The server ID
keyword

subscriptions

This is the subscriptions dataset of the STAN package, in charge of retrieving metrics about subscriptions from a STAN instance.

An example event for subscriptions looks as following:

{
    "@timestamp": "2024-06-18T10:43:38.692Z",
    "agent": {
        "ephemeral_id": "871b52f0-2644-4638-811c-1b0befe0ee13",
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "stan.subscriptions",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "8.0.0"
    },
    "elastic_agent": {
        "id": "21652d23-59b4-4d65-a341-ede15f919642",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "stan.subscriptions",
        "duration": 1947519984,
        "ingested": "2024-06-18T10:43:50Z",
        "module": "stan"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "8259e024976a406e8a54cdbffeb84fec",
        "ip": "192.168.252.5",
        "mac": "02-42-C0-A8-FC-05",
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "3.10.0-1160.102.1.el7.x86_64",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.6 LTS (Focal Fossa)"
        }
    },
    "metricset": {
        "name": "subscriptions",
        "period": 60000
    },
    "service": {
        "address": "http://elastic-package-service-stan-1:8222/streaming/channelsz?subs=1",
        "type": "stan"
    },
    "stan": {
        "cluster": {
            "id": "test-cluster"
        },
        "server": {
            "id": "SwiO2nzNV8CW27j45QPnFz"
        },
        "subscriptions": {
            "channel": "foo",
            "id": "benchmark-sub-23",
            "last_sent": 0,
            "offline": false,
            "pending": 0,
            "queue": "T",
            "stalled": false
        }
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields

FieldDescriptionTypeMetric Type
@timestamp
Event timestamp.
date
agent.id
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.
keyword
cloud.account.id
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.
keyword
cloud.availability_zone
Availability zone in which this host, resource, or service is located.
keyword
cloud.instance.id
Instance ID of the host machine.
keyword
cloud.provider
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.
keyword
cloud.region
Region in which this host, resource, or service is located.
keyword
container.id
Unique container id.
keyword
data_stream.dataset
Data stream dataset.
constant_keyword
data_stream.namespace
Data stream namespace.
constant_keyword
data_stream.type
Data stream type.
constant_keyword
event.dataset
Event dataset
constant_keyword
event.module
Event module
constant_keyword
host.name
Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.
keyword
service.address
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).
keyword
stan.cluster.id
The cluster ID
keyword
stan.server.id
The server ID
keyword
stan.subscriptions.channel
The name of the STAN channel the subscription is associated with
keyword
stan.subscriptions.id
The name of the STAN channel subscription (client_id)
keyword
stan.subscriptions.last_sent
Last known sequence number of the subscription that was acked
long
counter
stan.subscriptions.offline
Is the subscriber marked as offline?
boolean
stan.subscriptions.pending
Number of pending messages from / to the subscriber
long
gauge
stan.subscriptions.queue
The name of the NATS queue that the STAN channel subscription is associated with, if any
keyword
stan.subscriptions.stalled
Is the subscriber known to be stalled?
boolean

Changelog

VersionDetailsKibana version(s)

1.7.0

Enhancement View pull request
ECS version updated to 8.11.0. Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.

8.13.0 or higher

1.6.0

Enhancement View pull request
Add global filter on data_stream.dataset to improve performance.

8.10.2 or higher

1.5.0

Enhancement View pull request
Enable time series data for metrics data streams. This dramatically reduces storage for metrics and is expected to progressively improve query performance. For more details, see https://www.elastic.co/guide/en/elasticsearch/reference/current/tsds.html.

8.10.2 or higher

1.4.2

Enhancement View pull request
Migrate Client IP Count Timeline visualization to lens.

8.10.2 or higher

1.4.1

Enhancement View pull request
Add subscriptions data stream metric types mappings.

8.10.2 or higher

1.4.0

Enhancement View pull request
Add subscriptions data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.9

Enhancement View pull request
Add stats data stream metric types mappings.

8.10.2 or higher

1.3.8

Enhancement View pull request
Add stats data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.7

Enhancement View pull request
Add channels data stream metric types mappings.

8.10.2 or higher

1.3.6

Enhancement View pull request
Add channels data stream dimensions mappings and ECS fields.

8.10.2 or higher

1.3.5

Enhancement View pull request
Migrate Logs Overview dashboard visualizations to lens.

8.10.2 or higher

1.3.4

Enhancement View pull request
Migrate Metrics Overview dashboard visualizations to lens.

8.10.2 or higher

1.3.3

Bug fix View pull request
Fix reference error in Channel Overview dashboard.

7.14.0 or higher
8.0.0 or higher

1.3.2

Bug fix View pull request
Add null check to the rename processor

7.14.0 or higher
8.0.0 or higher

1.3.1

Enhancement View pull request
Added categories and/or subcategories.

7.14.0 or higher
8.0.0 or higher

1.3.0

Enhancement View pull request
Update to ECS 8.0

7.14.0 or higher
8.0.0 or higher

1.2.0

Enhancement View pull request
Release stan package for v8.0.0

7.14.0 or higher
8.0.0 or higher

1.1.2

Enhancement View pull request
Uniform with guidelines

—

1.1.1

Bug fix View pull request
Fix logic that checks for the 'forwarded' tag

—

1.1.0

Enhancement View pull request
Update to ECS 1.12.0

7.14.0 or higher

1.0.0

Enhancement View pull request
Release Stan as GA

—

0.5.3

Enhancement View pull request
Convert to generated ECS fields

—

0.5.2

Enhancement View pull request
update to ECS 1.11.0

—

0.5.1

Enhancement View pull request
Escape special characters in docs

—

0.5.0

Enhancement View pull request
Update integration description

—

0.4.0

Enhancement View pull request
Set "event.module" and "event.dataset"

—

0.3.0

Enhancement View pull request
update to ECS 1.10.0 and adding event.original options

—

0.2.0

Bug fix View pull request
Fix stack compatability

—

0.1.3

Enhancement View pull request
update to ECS 1.9.0

—

0.1.2

Bug fix View pull request
Change kibana.version constraint to be more conservative.

—

0.1.0

Enhancement View pull request
initial release

—

On this page