How to Send Data through Logstash or Kafka from Elastic APM | Elastic Blog
엔지니어링

Elastic APM에서 Logstash나 Kafka를 통해 데이터를 전송하는 방법

(/ VM, ) . .

6.4.0 APM Logstash Kafka , . , Logstash Logstash APM .

Elasticsearch .

  1. Elasticsearch Logstash
  2. Logstash Elastic APM
  3. Elasticsearch
  4. Kafka

Logstash

, beats/lumberjack APM Elasticsearch Logstash .

input {
   beats {
        id => "apm-server"
        port => 5044
  }
}
output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "apm-%{[beat][version]}-%{[processor][event]}-%{+YYYY.MM.dd}"
  }
}

APM (, ) . , .

, beat . APM Beats Framework . Beat Logstash APM [@metadata][beat] == "apm-server" .

APM

Logstash APM 'apm-server.yml' .

output.elasticsearch:
  enabled: false
output.logstash
  enabled: true
  hosts: ["logstash:5044"]

, ssl Elastic APM .

APM , .

xpack.monitoring.elasticsearch:
  enabled: true
  hosts: ["elasticsearch:9200"]

APM Elasticsearch .

Elasticsearch

APM . Elasticsearch APM UI APM . 릿 . 릿 .

Elasticsearch APM , .

 apm-server setup --template

, APM 릿 .

apm-server export template > apm-6.4.2.tmpl.json

Elasticsearch . Curl .

curl -XPUT -H 'Content-Type: application/json' http://elasticsearch:9200/_template/apm-6.4.2 -d @apm-6.4.2.tmpl.json

. 릿 _template/apm-* . .

{
  "apm-6.4.2": {
    "order": 1,
    "index_patterns": [
      "apm-6.4.2-*"
    ],
...

Logstash . , . , .

  1. Logstash APM Server .
  2. APM .

APM .

. .

APM Elasticsearch . APM Elasticsearch . 'rum.source_mapping.elasticsearch' .

apm-server:
  rum:
    source_mapping:
      elasticsearch:
        hosts: ["elasticsearch:9200"]
        index_pattern: "apm-*-sourcemap*"

APM apm--sourcemap" .

Logstash , Elasticsearch . , Logstash , .

Kafka

Kafka Elasticsearch APM . Kafka APM .

output.kafka:
  enabled: true
  hosts: ["kafka:9092"]
  topics:
    - default: 'apm'
      topic: 'apm-%{[context.service.name]}'

, . .

Kafka Logstash Elasticsearch . :

input {
  kafka {
    id => "apm-server-kafka"
    bootstrap_servers => ["kafka:9092"]
    topics_pattern => "apm.*"
    codec => "json"
  }
}
output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "apm-%{[beat][version]}-%{[processor][event]}-%{+YYYY.MM.dd}"
  }
}

, APM (, ) . Logstash Kafka .

APM Logstash Kafka APM . . . ] .