<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
  <channel>
    <title><![CDATA[Peter Simkins - Elastic Observability Labs]]></title>
    <description><![CDATA[Trusted security news & research from the team at Elastic.]]></description>
    <copyright><![CDATA[© 2026. Elasticsearch B.V. All Rights Reserved]]></copyright>
    <image>
      <title><![CDATA[Peter Simkins - Elastic Observability Labs]]></title>
      <url>https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltad972c1c27dbefc6/6a88d9782904ea5e8511d473/observability-labs-thumbnail.png</url>
      <link>https://www.elastic.co/observability-labs/author/peter-simkins</link>
    </image>
    <link>https://www.elastic.co/observability-labs/author/peter-simkins</link>
    <atom:link href="https://www.elastic.co/observability-labs/rss/author/peter-simkins.xml" rel="self" type="application/rss+xml"/>
    <language><![CDATA[en]]></language>
    <lastBuildDate>Tue, 22 Sep 2026 04:22:56 GMT</lastBuildDate>
  <item>
    <title><![CDATA[Migrate Datadog Kubernetes dashboards to Elastic Observability in under an hour]]></title>
    <description><![CDATA[See how the migration CLI translates a real Datadog Kubernetes dashboard into validated Kibana panels and uploads it to your cluster in under an hour, no manual widget rebuilds required.]]></description>
    <content:encoded><![CDATA[<p>The <a href="https://github.com/elastic/observability-migration-platform">Observability Migration Platform</a> takes a Datadog Kubernetes dashboard and turns it into ES|QL-backed Lens panels in Kibana. It validates queries against your live cluster before upload, and the whole process typically fits in under an hour. This walkthrough uses the <strong>Kubernetes - Overview</strong> board: pod CPU, working set memory, pod phases, and CrashLoopBackOff counts. Elasticsearch runs ES|QL time series queries up to 30× faster than Prometheus on common gauge and counter workloads in published benchmarks, with up to 2.5× better storage efficiency. Review the migration report and enable alerts when you are ready.</p>
<h2 id="thedatadogkubernetesdashboardusedinthismigration">The Datadog Kubernetes dashboard used in this migration</h2>
<p>The walkthrough uses <strong>Kubernetes - Overview</strong> from <code>infra/datadog/dashboards/integrations/kubernetes.json</code> in the migration repository. It is a cluster-wide board with the signals operators check during an incident: pod counts, CPU and memory by host or pod, non-running pods, and containers stuck in CrashLoopBackOff.</p>
<p>Below are representative queries from the source dashboard:</p>
<pre><code># Pod CPU by host
sum:kubernetes.cpu.usage.total{$scope,$cluster,$label,$node} by {host}
</code></pre>
<pre><code># Pod memory by pod
sum:kubernetes.memory.usage{$scope,$deployment,$statefulset,$replicaset,$daemonset,$cluster,$namespace,!pod_name:no_pod,$label,$service,$node} by {pod_name}
</code></pre>
<pre><code># Pods not running (pressure / scheduling signal)
sum:kubernetes_state.pod.status_phase{$scope,$cluster,$namespace,$deployment,$statefulset,$replicaset,$daemonset,!pod_phase:running,!pod_phase:succeeded,$label,$node,$service} by {kube_cluster_name,kube_namespace,pod_phase}
</code></pre>
<pre><code># CrashLoopBackOff
sum:kubernetes_state.container.status_report.count.waiting{$cluster,$namespace,$deployment,$statefulset,$replicaset,$daemonset,reason:crashloopbackoff,$scope,$daemonset,$label,$node,$service} by {pod_name}
</code></pre>
<p>If this board translates cleanly, most production Datadog Kubernetes folders are worth testing with the same workflow.</p>
<h2 id="whydatadogtoelasticmigrationisfasternow">Why Datadog-to-Elastic migration is faster now</h2>
<p>The migration platform automates the query translation and panel rebuilds that used to dominate Datadog moves. Elasticsearch stores Kubernetes metrics efficiently and runs the ES|QL queries those panels use. See <a href="https://www.elastic.co/observability-labs/blog/prometheus-metrics-elasticsearch-faster-cheaper-datadog">Elasticsearch as a metrics backend</a> for benchmark context and storage comparisons.</p>
<p>The platform maps Datadog queries to Kibana panels, validates ES|QL against live data, and writes artifacts you can inspect before anything goes to production.</p>
<h2 id="prerequisites">Prerequisites</h2>
<p>You need an <a href="https://www.elastic.co/docs/solutions/observability/get-started">Elastic Observability Serverless</a> project, a <a href="https://www.elastic.co/docs/deploy-manage/api-keys/serverless-project-api-keys">project API key</a>, and the migration CLI installed from the <a href="https://github.com/elastic/observability-migration-platform">Observability Migration Platform</a> repository.</p>
<p>Export your endpoints and API key:</p>
<pre><code>export ELASTICSEARCH_ENDPOINT="https://YOUR_ES_ENDPOINT"
export KIBANA_ENDPOINT="https://YOUR_KIBANA_ENDPOINT"
export KEY="YOUR_API_KEY"
</code></pre>
<p>Install the CLI and confirm the toolchain:</p>
<pre><code>python3 -m venv .venv
.venv/bin/pip install ".[all]"
.venv/bin/obs-migrate doctor
</code></pre>
<p>The <code>doctor</code> command checks compile and lint dependencies. Resolve any errors before you migrate production dashboards. Pin a release tag if you plan to run this in CI.</p>
<p>To pull dashboards from the Datadog API instead of JSON files, copy <code>datadog_creds.env.example</code> to <code>datadog_creds.env</code> and set <code>DD_API_KEY</code>, <code>DD_APP_KEY</code>, and <code>DD_SITE</code>.</p>
<h2 id="ingestkubernetesmetricsfirst">Ingest Kubernetes metrics first</h2>
<p>Empty panels after upload usually mean Elasticsearch does not yet have the series the Datadog queries reference. Make sure to confirm ingest before you run the migration.</p>
<p>There are two common paths to do so:</p>
<ol>
<li>OpenTelemetry into managed OTLP with Kubernetes receivers (<code>kubeletstats</code>, <code>k8s_cluster</code>), then explore in Discover</li>
<li>Existing Prometheus or agent pipelines that already write pod and node metrics to <code>metrics-*</code></li>
</ol>
<p>The migration CLI accepts <code>--field-profile otel</code> to map Datadog tags such as <code>pod_name</code>, <code>kube_namespace</code>, and <code>kube_cluster_name</code> to OpenTelemetry fields like <code>kubernetes.pod.name</code> and <code>kubernetes.namespace</code>. If panels are empty after migration, verify field mapping and the selected time range before you change translator settings.</p>
<p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt9d1b8627fecca77c/6a85cd129a32f11916a7e026/metrics-exploration.jpg" alt="Kubernetes metrics exploration in Discover with live CPU and memory charts after OpenTelemetry ingest" /></p>
<h2 id="runthedatadogdashboardmigrationcli">Run the Datadog dashboard migration CLI</h2>
<p>Export the Datadog dashboard JSON from the UI, or copy the sample <code>kubernetes.json</code> from <code>infra/datadog/dashboards/integrations/</code> in the migration repo. Place files in a directory such as <code>./datadog_k8s_exports/</code>.</p>
<p>Run the migration from that directory:</p>
<pre><code>datadog-migrate \
  --source files \
  --input-dir ./datadog_k8s_exports \
  --output-dir ./migration_output \
  --assets all \
  --field-profile otel \
  --data-view "metrics-*" \
  --logs-index "logs-*" \
  --upload \
  --kibana-url "$KIBANA_ENDPOINT" \
  --kibana-api-key "$KEY" \
  --ensure-data-views \
  --create-alert-rules \
  --validate \
  --es-url "$ELASTICSEARCH_ENDPOINT" \
  --es-api-key "$KEY"
</code></pre>
<p>These flags matter for Kubernetes boards:</p>
<ul>
<li><code>--field-profile otel</code> maps Datadog Kubernetes fields to OpenTelemetry field names in Elasticsearch</li>
<li><code>--assets all</code> includes dashboards and Datadog monitor definitions when present</li>
<li><code>--validate</code> runs emitted ES|QL against your cluster before upload</li>
<li><code>--create-alert-rules</code> creates Kibana rules in a disabled state</li>
</ul>
<p>The unified CLI performs the same work:</p>
<pre><code>obs-migrate migrate \
  --source datadog \
  --input-mode files \
  --input-dir ./datadog_k8s_exports \
  --output-dir ./migration_output \
  --assets all \
  --field-profile otel \
  --data-view "metrics-*" \
  --logs-index "logs-*" \
  --validate \
  --es-url "$ELASTICSEARCH_ENDPOINT" \
  --es-api-key "$KEY" \
  --kibana-url "$KIBANA_ENDPOINT" \
  --kibana-api-key "$KEY" \
  --upload \
  --create-alert-rules
</code></pre>
<p>To fetch a dashboard from Datadog directly:</p>
<pre><code>datadog-migrate \
  --source api \
  --env-file datadog_creds.env \
  --dashboard-ids YOUR_DASHBOARD_ID \
  --output-dir ./migration_output \
  --assets all \
  --field-profile otel \
  --data-view "metrics-*" \
  --validate \
  --es-url "$ELASTICSEARCH_ENDPOINT" \
  --es-api-key "$KEY" \
  --kibana-url "$KIBANA_ENDPOINT" \
  --kibana-api-key "$KEY" \
  --upload
</code></pre>
<p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltb0d737676b7d2089/6a85cd1543c0b782232f065e/migration-flow.png" alt="End-to-end Observability Migration Platform flow from Datadog extract through translate, validate, compile, and upload to Kibana" /></p>
<h2 id="validatethemigrateddatadogdashboardinkibana">Validate the migrated Datadog dashboard in Kibana</h2>
<p>Open Kibana → <strong>Dashboards</strong> and locate the migrated <strong>Kubernetes - Overview</strong> board. Confirm that cluster and namespace pod counts, CPU and memory series, pod phase panels, CrashLoopBackOff widgets, and deployment replica charts return data for your selected time range.</p>
<p>If you migrated monitors, open <strong>Observability → Rules</strong>. Imported rules remain disabled until you enable them after reviewing thresholds.</p>
<p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltab0ca21249e6b216/6a85cd17078290f6ba3217a4/kibana-k8s-overview.jpg" alt="Kubernetes overview dashboard in Kibana with cluster and node CPU, memory, and readiness views after Datadog migration" /></p>
<p>The CLI also writes local artifacts under <code>./migration_output/</code>:</p>
<ul>
<li><code>dashboards/yaml/</code> contains the translated dashboard definition.</li>
<li><code>dashboards/migration_report.json</code> lists panels that translated automatically and panels flagged for manual review.</li>
<li><code>alerts/</code> contains monitor translations when monitors were included in the export.</li>
</ul>
<h2 id="handlemanualreviewpanels">Handle manual-review panels</h2>
<p>Some Datadog widget types do not translate on the first pass. Exotic formulas, log-only panels, and unsupported widgets appear as manual-review entries in the migration report rather than as silently broken charts.</p>
<p>| Result | Recommended action |
| --- | --- |
| Panel returns data | Accept the translation and continue |
| Panel is empty | Confirm metric names and <code>data_stream.dataset</code> values in <code>metrics-*</code>, then widen or shift the time range |
| Manual-review marker | Open the original Datadog query and simplify or redesign the panel |
| Monitor never fires | Confirm the rule is enabled and thresholds match your environment |</p>
<p>Datadog coverage is narrower than Grafana in some areas. Read the migration report before you commit to full parity with leadership. The platform prefers conservative failures over uploading panels that look correct but query the wrong fields.</p>
<h2 id="relateddatadogandgrafanamigrationguides">Related Datadog and Grafana migration guides</h2>
<p>For the Grafana PromQL version of this workflow, see <a href="https://www.elastic.co/observability-labs/blog/grafana-elastic-kubernetes-dashboard-migration">Migrate your Grafana Kubernetes dashboard to Elastic Observability</a>. For platform-level context, see <a href="https://www.elastic.co/observability-labs/blog/migrate-datadog-grafana-dashboards-alerts-to-kibana">Migrating Datadog and Grafana dashboards and alerts to Kibana</a>. Review <a href="https://github.com/elastic/observability-migration-platform/blob/main/docs/known-limitations.md">known limitations</a> before you migrate every production folder.</p>]]></content:encoded>
    <link>https://www.elastic.co/observability-labs/blog/datadog-kubernetes-dashboard-migration</link>
    <guid isPermaLink="false">datadog-kubernetes-dashboard-migration</guid>
    <category><![CDATA[Kubernetes]]></category>
    <category><![CDATA[Metrics]]></category>
    <category><![CDATA[Data Management]]></category>
    <dc:creator><![CDATA[Peter Simkins]]></dc:creator>
    <enclosure url="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltb63e39bf3a11e972/6a85cd1a9bf994ca880a05af/header.jpg" length="0" type="image/jpeg"/>
    <pubDate>Thu, 23 Jul 2026 00:00:00 GMT</pubDate>
  </item>
  <item>
    <title><![CDATA[Migrate your Grafana Kubernetes dashboard to Elastic Observability: same PromQL, 30x faster queries]]></title>
    <description><![CDATA[Take a real Grafana Kubernetes dashboard covering pod CPU, memory, node pressure, and restart counts, then migrate it into Elastic Observability with native PromQL in under an hour.]]></description>
    <content:encoded><![CDATA[<p>Elasticsearch now runs PromQL natively. Migrate a Grafana <strong>Kubernetes / Views / Global</strong> dashboard into <a href="https://www.elastic.co/docs/solutions/observability">Elastic Observability</a> with the <a href="https://github.com/elastic/observability-migration-platform">Observability Migration Platform</a>. The sample board covers pod CPU, working set memory, throttling signals, and container restart counts. With Kubernetes metrics already in Elasticsearch, the translation, validation, and upload steps typically fit in under an hour.</p>
<p>The migration tool keeps panel queries in PromQL instead of rewriting them into a new dialect, validates them when you pass <code>--validate</code>, and uploads compiled dashboards to Kibana. You still review the migration report and enable alerts on your schedule.</p>
<h2 id="samplegrafanakubernetesdashboardusedinthismigration">Sample Grafana Kubernetes dashboard used in this migration</h2>
<p>The walkthrough uses <strong>Kubernetes / Views / Global</strong>, a community-style Grafana dashboard in the migration repository. It includes the signals operators check during an incident: namespace CPU and memory, throttling pressure, and restart counts.</p>
<p>Below are representative PromQL queries from the source dashboard:</p>
<pre><code># Pod / container CPU by namespace
sum(rate(container_cpu_usage_seconds_total{image!="", cluster="$cluster"}[$__rate_interval])) by (namespace)
</code></pre>
<pre><code># Memory working set by namespace
sum(container_memory_working_set_bytes{image!="", cluster="$cluster"}) by (namespace)
</code></pre>
<pre><code># Node / CPU pressure style signal: throttled seconds
sum(rate(container_cpu_cfs_throttled_seconds_total{image!="", cluster="$cluster"}[$__rate_interval])) by (namespace) &gt; 0
</code></pre>
<pre><code># Container restart counts
sum(increase(kube_pod_container_status_restarts_total{cluster="$cluster"}[$__rate_interval])) by (namespace) &gt; 0
</code></pre>
<p>If this board translates cleanly, most production Grafana Kubernetes folders are worth testing with the same workflow.</p>
<h2 id="whygrafanatoelasticmigrationisfasternow">Why Grafana-to-Elastic migration is faster now</h2>
<p>The migration platform automates the query translation and panel rebuilds that used to dominate Grafana moves. Native <a href="https://www.elastic.co/observability-labs/blog/prometheus-remote-write-elasticsearch">Prometheus Remote Write</a> and <a href="https://www.elastic.co/observability-labs/blog/elasticsearch-supports-promql">PromQL in Kibana</a> let you keep the dialect your on-call team already uses. The <code>--native-promql</code> flag passes PromQL through unchanged. See <a href="https://www.elastic.co/observability-labs/blog/prometheus-metrics-elasticsearch-faster-cheaper-datadog">Elasticsearch as a metrics engine</a> for storage and query context.</p>
<h2 id="prerequisites">Prerequisites</h2>
<p>You need an <a href="https://www.elastic.co/docs/solutions/observability/get-started">Elastic Observability Serverless</a> project, a <a href="https://www.elastic.co/docs/deploy-manage/api-keys/serverless-project-api-keys">project API key</a>, and the migration CLI installed from the <a href="https://github.com/elastic/observability-migration-platform">observability-migration-platform</a> repository.</p>
<p>Export your endpoints and API key:</p>
<pre><code>export ELASTICSEARCH_ENDPOINT="https://YOUR_ES_ENDPOINT"
export KIBANA_ENDPOINT="https://YOUR_KIBANA_ENDPOINT"
export KEY="YOUR_API_KEY"
</code></pre>
<p>Install the CLI and confirm the toolchain:</p>
<pre><code>python3 -m venv .venv
.venv/bin/pip install ".[all]"
.venv/bin/obs-migrate doctor
</code></pre>
<p>The <code>doctor</code> command checks compile and lint dependencies. Resolve any errors before you migrate production dashboards. Pin a release tag if you plan to run this in CI.</p>
<h2 id="howdoyougetkubernetesmetricsintoelasticsearch">How do you get Kubernetes metrics into Elasticsearch?</h2>
<p>Empty panels after upload usually mean Elasticsearch does not yet have the series the PromQL references. Make sure to confirm ingest before you run the migration.</p>
<p>There are two common paths to do so:</p>
<ol>
<li><a href="https://www.elastic.co/observability-labs/blog/prometheus-remote-write-elasticsearch">Prometheus Remote Write into Elasticsearch</a> from kube-state-metrics, cAdvisor or kubelet metrics, and node exporters.</li>
<li>OpenTelemetry into managed OTLP for Kubernetes receivers, then explore in Discover.</li>
</ol>
<p>Test with a query such as <code>sum(rate(container_cpu_usage_seconds_total[5m])) by (namespace)</code> against Elastic. If that returns data, continue. If it does not, fix ingest first.</p>
<p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt0e2a3acc12c55296/6a85cd31f61d6e81da9c2b5f/metrics-exploration.jpg" alt="Kubernetes metrics exploration in Discover with live CPU and memory charts" /></p>
<h2 id="runthegrafanadashboardmigrationcli">Run the Grafana dashboard migration CLI</h2>
<p>Export your Grafana dashboard JSON, or copy the sample <code>k8s-views-global.json</code> from <code>infra/grafana/dashboards/</code> in the migration repo. Place files in a directory such as <code>./grafana_k8s_exports/</code>.</p>
<p>Run the migration from that directory:</p>
<pre><code>grafana-migrate \
  --source files \
  --input-dir ./grafana_k8s_exports \
  --output-dir ./migration_output \
  --assets all \
  --native-promql \
  --data-view "metrics-*" \
  --esql-index "metrics-*" \
  --upload \
  --kibana-url "$KIBANA_ENDPOINT" \
  --kibana-api-key "$KEY" \
  --ensure-data-views \
  --create-alert-rules \
  --validate \
  --es-url "$ELASTICSEARCH_ENDPOINT" \
  --es-api-key "$KEY"
</code></pre>
<p>These flags matter for Kubernetes boards:</p>
<ul>
<li><code>--native-promql</code> keeps pod CPU, memory, throttling, and restart queries in PromQL</li>
<li><code>--assets all</code> includes dashboards and Grafana PromQL alert definitions when present</li>
<li><code>--validate</code> runs emitted queries against Elasticsearch before upload</li>
<li><code>--create-alert-rules</code> creates Kibana rules in a disabled state</li>
</ul>
<p>The unified CLI performs the same work:</p>
<pre><code>obs-migrate migrate \
  --source grafana \
  --input-mode files \
  --input-dir ./grafana_k8s_exports \
  --output-dir ./migration_output \
  --assets all \
  --native-promql \
  --data-view "metrics-*" \
  --validate \
  --es-url "$ELASTICSEARCH_ENDPOINT" \
  --es-api-key "$KEY" \
  --kibana-url "$KIBANA_ENDPOINT" \
  --kibana-api-key "$KEY" \
  --upload \
  --create-alert-rules
</code></pre>
<p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltb2b8eab23dc07def/6a85cd349d2b71883df939e6/migration-flow.png" alt="End-to-end Observability Migration Platform flow from Grafana extract through translate, validate, compile, and upload to Kibana" /></p>
<h2 id="validatethemigratedgrafanadashboardinkibana">Validate the migrated Grafana dashboard in Kibana</h2>
<p>Open Kibana → <strong>Dashboards</strong> and locate <strong>Kubernetes / Views / Global</strong>. Confirm that namespace or pod CPU utilization, memory working set panels, throttling or pressure widgets, and restart charts return data for your selected time range.</p>
<p>If you migrated alerts, open <strong>Observability → Rules</strong>. Imported rules remain disabled until you enable them after reviewing thresholds.</p>
<p><img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt43d391e3d324bed2/6a85cd37501a858109fbb37b/kibana-k8s-overview.jpg" alt="Kubernetes overview dashboard in Kibana with cluster and node CPU, memory, and readiness views after Grafana migration" /></p>
<p>The CLI also writes local artifacts under <code>./migration_output/</code>:</p>
<ul>
<li><code>dashboards/yaml/</code> contains the translated dashboard definition.</li>
<li><code>dashboards/migration_report.json</code> lists panels that translated automatically and panels flagged for manual review.</li>
<li><code>alerts/</code> contains alert translations when alert definitions were included in the export.</li>
</ul>
<h2 id="whatdoyoudowhengrafanapanelsdontmigrateautomatically">What do you do when Grafana panels don't migrate automatically?</h2>
<p>The Observability Migration Platform flags PromQL expressions that do not translate automatically. Hard joins, unusual arithmetic, and a few Alertmanager-era edge cases appear as manual-review entries in the migration report rather than as silently broken charts.</p>
<p>| Result | Recommended action |
| --- | --- |
| Panel returns data | Accept the translation and continue |
| Panel is empty | Confirm metric names exist in <code>metrics-*</code>, then widen or shift the time range |
| Manual-review marker | Open the original PromQL and simplify or redesign the panel |
| Alert never fires | Confirm the rule is enabled and thresholds match your environment |</p>
<p>This path migrates Grafana PromQL dashboards and Grafana unified PromQL alert definitions into Kibana. It does not ingest a raw <code>alertmanager.yml</code>. The goal is to keep the PromQL your pager already trusts instead of rebuilding the Kubernetes board from zero.</p>
<h2 id="relatedguides">Related guides</h2>
<p>For platform-level context, see <a href="https://www.elastic.co/observability-labs/blog/migrate-datadog-grafana-dashboards-alerts-to-kibana">Migrating Datadog and Grafana dashboards and alerts to Kibana</a>. Review <a href="https://github.com/elastic/observability-migration-platform/blob/main/docs/known-limitations.md">known limitations</a> before you migrate every production folder.</p>]]></content:encoded>
    <link>https://www.elastic.co/observability-labs/blog/grafana-elastic-kubernetes-dashboard-migration</link>
    <guid isPermaLink="false">grafana-elastic-kubernetes-dashboard-migration</guid>
    <category><![CDATA[Kubernetes]]></category>
    <category><![CDATA[Metrics]]></category>
    <dc:creator><![CDATA[Peter Simkins]]></dc:creator>
    <enclosure url="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltb0d0d5c14499f847/6a85cd3a331d7a6951c317f1/header.jpg" length="0" type="image/jpeg"/>
    <pubDate>Wed, 22 Jul 2026 00:00:00 GMT</pubDate>
  </item>
  </channel>
</rss>