メトリック分析
ドローンセンサーのデータからCPUの使用率まで、Elasticsearchの優れた全文検索機能を活かしてメトリック情報を収集します。進化する分析モデルを体感してください。
あらゆるデータを収集、分析して異常を検知。はじめましょう
データソースが多数、分析は１か所で
監視対象がDockerコンテナーでも、データセンターでも、大量のロボットでも、火星の表面温度でも、Beatsとモジュールとプラグインで投入して完了。それがElastic Stackです。
では、このすべてのデータを1つの検索エンジンで扱うことはできる...？もうおわかりですね。
はじめてみましょう。
その数値を、自在に探索
ディメンション、タグ、データ数、フィールドを縦横無尽に分析しましょう。どのような制限もありません。ホスト名、IPアドレス、デプロイメント、配色などは好みに応じてカスタマイズできます。 収集、パースからタグ付けまで、処理はBeatsとモジュールにおまかせ。ダッシュボードや機械学習ジョブを作成することもできます。
スピードとスケールを体感
転置インデックスを上回る機能が誕生した背景には、新しいデータタイプの作成、BKDツリーの実装、カラムストアの追加といった工夫があります。高速な検索、メモリやディスク使用量の削減、より効率的なデータの構造化といったメリットを実現しています。つまり、ペタバイト規模のデータフィールドや値へ高速にアクセスすることができます。
適切なロールアップ
Elastic Stackで、メトリックデータをダウンサンプリングできます。ElasticsearchとKibanaのロールアップ機能は容量を節約してクエリを高速化しながら、高い精度を保ちます。
はじめましょう
- Register, if you do not already have an account. Free 14-day trial available.
- Log into the Elastic Cloud console
- Select Create Deployment, and specify the Deployment Name
- Modify the other deployment options as needed (or not, the defaults are great to get started)
- Click Create Deployment
- Save the Cloud ID and the cluster Password for your records, we will refer to these as <cloud.id> and <password> below
- Wait until deployment creation completes
Download and unpack Metricbeat
Open terminal (varies depending on your client OS) and in the Metricbeat install directory, type:
Paste in the <password> for the elastic user when prompted
Paste in the <cloud.id> for the cluster when prompted
To modify defaults, edit modules.d/system.yml.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing CPU and memory utilization, and process-level statistics.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
- Register, if you do not already have an account. Free 14-day trial available.
- Log into the Elastic Cloud console
- Select Create Deployment, and specify the Deployment Name
- Modify the other deployment options as needed (or not, the defaults are great to get started)
- Click Create Deployment
- Save the Cloud ID and the cluster Password for your records, we will refer to these as <cloud.id> and <password> below
- Wait until deployment creation completes
Download and unpack Metricbeat
Open terminal (varies depending on your client OS) and in the Metricbeat install directory, type:
Paste in the <password> for the elastic user when prompted
Paste in the <cloud.id> for the cluster when prompted
To modify defaults, edit modules.d/apache.yml.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing connection statistics and HTTP worker details.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
- Register, if you do not already have an account. Free 14-day trial available.
- Log into the Elastic Cloud console
- Select Create Deployment, and specify the Deployment Name
- Modify the other deployment options as needed (or not, the defaults are great to get started)
- Click Create Deployment
- Save the Cloud ID and the cluster Password for your records, we will refer to these as <cloud.id> and <password> below
- Wait until deployment creation completes
Download and unpack Metricbeat
Open terminal (varies depending on your client OS) and in the Metricbeat install directory, type:
Paste in the <password> for the elastic user when prompted
Paste in the <cloud.id> for the cluster when prompted
To modify defaults, edit modules.d/mongodb.yml.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing data statistics, health and status information about your MongoDB deployment.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
- Register, if you do not already have an account. Free 14-day trial available.
- Log into the Elastic Cloud console
- Select Create Deployment, and specify the Deployment Name
- Modify the other deployment options as needed (or not, the defaults are great to get started)
- Click Create Deployment
- Save the Cloud ID and the cluster Password for your records, we will refer to these as <cloud.id> and <password> below
- Wait until deployment creation completes
Download and unpack Metricbeat
Open terminal (varies depending on your client OS) and in the Metricbeat install directory, type:
Paste in the <password> for the elastic user when prompted
Paste in the <cloud.id> for the cluster when prompted
To modify defaults, edit modules.d/docker.yml.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing data statistics, health and status information about your Docker deployment.Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
- Register, if you do not already have an account. Free 14-day trial available.
- Log into the Elastic Cloud console
- Select Create Deployment, and specify the Deployment Name
- Modify the other deployment options as needed (or not, the defaults are great to get started)
- Click Create Deployment
- Save the Cloud ID and the cluster Password for your records, we will refer to these as <cloud.id> and <password> below
- Wait until deployment creation completes
Download and unpack Metricbeat
Open terminal (varies depending on your client OS) and in the Metricbeat install directory, type:
Paste in the <password> for the elastic user when prompted
Paste in the <cloud.id> for the cluster when prompted
From your machine or wherever you run kubectl:
- Download metricbeat-kubernetes.yml
- Edit metricbeat-kubernetes.yml to point to your Elasticsearch instance
env:
- name: ELASTIC_CLOUD_ID
value: <cloud.id>
- name: ELASTIC_CLOUD_AUTH
value: elastic:<cloud.auth>
Replace <cloud.id> and <cloud.auth> with the values that you saved when you created a deployment in the step “Set up Elastic Cloud”
Optionally, you can enable kube-state-metrics for more detail.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can monitor your Kubernetes cluster.Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
- Register, if you do not already have an account. Free 14-day trial available.
- Log into the Elastic Cloud console
- Select Create Deployment, and specify the Deployment Name
- Modify the other deployment options as needed (or not, the defaults are great to get started)
- Click Create Deployment
- Save the Cloud ID and the cluster Password for your records, we will refer to these as <cloud.id> and <password> below
- Wait until deployment creation completes
Download and unpack Heartbeat (Beta)
Open terminal (varies depending on your client OS) and in the Heartbeat install directory, type:
Paste in the <password> for the elastic user when prompted
Paste in the <cloud.id> for the cluster when prompted
What just happened?
Heartbeat is designed to do distributed uptime checks from each of your hosts to ensure that they can each reach every endpoint they are supposed to. This is amazing for service-oriented architectures. In this case, you've asked Heartbeat to check the uptime for the two local ports corresponding to the Elasticsearch and Kibana defaults. Heartbeat then sends this data to Elasticsearch and you can see the data in the Kibana dashboard.
Didn't work for you?
Heartbeat was set to use the default ports for Elasticsearch and Kibana in this example. See the documentation for configuration options.
To modify defaults, edit modules.d/system.yml.
Start a trial to experience Machine Learning or enable authentication.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing CPU and memory utilization, and process-level statistics.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
To modify defaults, edit modules.d/apache.yml.
Start a trial to experience Machine Learning or enable authentication.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing connection statistics and HTTP worker details.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
To modify defaults, edit modules.d/mongodb.yml.
Start a trial to experience Machine Learning or enable authentication.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing data statistics, health and status information about your MongoDB deployment.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
To modify defaults, edit modules.d/docker.yml.
Start a trial to experience Machine Learning or enable authentication.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can start viewing data statistics, health and status information about your Docker deployment.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
- Download metricbeat-kubernetes.yml
- Edit metricbeat-kubernetes.yml and specify the host for your Elasticsearch server (If you are connecting back to your host from kubernetes running locally then set ELASTICSEARCH_HOST to host.docker.internal). There is a DaemonSet and a singleton, edit the HOST for both:
- name: ELASTICSEARCH_HOST
value: host.docker.internal
Optionally, you can enable kube-state-metrics for more detail.
Start a trial to experience Machine Learning or enable authentication.
What just happened?
Metricbeat created an index pattern in Kibana with defined fields, searches, visualizations, and dashboards. In a matter of minutes you can monitor your Kubernetes cluster.
Didn't work for you?
Metricbeat modules have defaults and configurations for each system they connect to. See the documentation for supported versions and configuration options.
Start a trial to experience Machine Learning or enable authentication.
What just happened?
Heartbeat is designed to do distributed uptime checks from each of your hosts to ensure that they can each reach every endpoint they are supposed to. This is amazing for service-oriented architectures. In this case, you've asked Heartbeat to check the uptime for the two local ports corresponding to the Elasticsearch and Kibana defaults. Heartbeat then sends this data to Elasticsearch and you can see the data in the Kibana dashboard.
Didn't work for you?
Heartbeat was set to use the default ports for Elasticsearch and Kibana in this example. See the documentation for configuration options.
機械学習で、異常を検知する
データ量が大きくなるほど、ストリーミングの平均値や測定値、合計値の異常を発見することがむずかしくなります。常に全てのデータを監視しようとすることは、現実的ではありません。
Elastic Stackの機械学習機能は、膨大なデータの中から自動的に異常を検知します。何が「通常」か分析し、学習することで「異常」を見つける仕組みです。
超大規模データにも対応
米国国立エネルギー研究科学コンピューティングセンター （NERSC）では、一日に12億ドキュメント、160GBのデータをElastic Stackで分析しています。変電所の電力使用率のKPIから建物の空気や水の温度、コンピューターディスク、ネットワークI/O、システム負荷まで。あらゆる種類のメトリックをインデックスして、科学を進歩させる取り組みです。
Elastic Stackの導入事例はこれだけではありません。 他のユーザー事例も併せてご覧ください。
メトリック分析をはじめましょう
ネットワークデータやインフラのログ、膨大なテキストドキュメントも、メトリックとともにElastic Stackに投入しましょう。分析の幅を広げ、ワークフローを最適化しながら、アーキテクチャをシンプルにすることができます。
