Loading

Add internal telemetry to an OTel Collector monitored by Fleet

Monitor the health, resource usage, and pipeline performance of an OpenTelemetry (OTel) Collector in Kibana by sending its own metrics, logs, and traces to Elasticsearch. The collector emits this telemetry to its own OTLP (OpenTelemetry Protocol) receiver, which forwards it through a pipeline to Elasticsearch.

Note

If you added your collector using the Add collector flow, the generated configuration already includes internal telemetry.

You'll need:

  • An OTel Collector with OpAMP extension support, already added in Fleet. For more information, refer to Add an OTel Collector in Fleet.
  • Access to your OTel Collector's configuration file
  • An Elasticsearch API key with create_index, write, and auto_configure index privileges on metrics-*, logs-*, and traces-* data streams, or an existing exporter you can reuse for internal telemetry

To add internal telemetry to an existing collector configuration, extend it with the following components:

  1. Configure the OTLP receiver:

    receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

  2. Configure an exporter that sends telemetry to your Elasticsearch backend. If your collector already exports telemetry, you can reuse the existing exporter and add the OTLP receiver to its pipelines. Otherwise, configure the elasticsearch/otel exporter:

    exporters:
  elasticsearch/otel:
    endpoints: [https://elasticsearch:9200]
    api_key: "<es-api-key>"
    mapping:
      mode: otel
    1. Replace with your Elasticsearch endpoint.
    2. Replace with an Elasticsearch API key.
    Note

    On Serverless Observability projects and Elastic Cloud Hosted deployments, you can replace the elasticsearch/otel exporter with the Elastic Cloud Managed OTLP Endpoint, which accepts OTLP data directly and uses an APM-scoped API key. For more details, refer to Elastic Cloud Managed OTLP Endpoint (mOTLP).

  3. Set up the service pipelines:

    service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [elasticsearch/otel]
    logs:
      receivers: [otlp]
      exporters: [elasticsearch/otel]
    traces:
      receivers: [otlp]
      exporters: [elasticsearch/otel]

  4. Set up internal telemetry for the OTel Collector:

    service:
  telemetry:
    resource:
      service.instance.id: "<instance-uid>"
    metrics:
      level: detailed
      readers:
        - periodic:
            interval: 3000
            exporter:
              otlp:
                protocol: grpc
                endpoint: http://localhost:4317
    logs:
      processors:
        - batch:
            exporter:
              otlp:
                protocol: grpc
                endpoint: http://localhost:4317
    traces:
      processors:
        - batch:
            exporter:
              otlp:
                protocol: grpc
                endpoint: http://localhost:4317
    1. Replace with the UUID value provided in extensions.opamp.instance_uid.

  5. Save your configuration, then start or restart the OTel Collector.

When your collector is running, confirm that internal telemetry is reaching Elasticsearch:

  1. The first time the collector ingests data, Fleet automatically installs the OTel Collector internal telemetry dashboards in Kibana.
  2. In Kibana, go to Dashboards and search for OTel Collector internal telemetry.
  3. Open a dashboard and confirm that visualizations display your collector's metrics, logs, traces, resource usage, pipeline performance, and component health.