<p>The Unix world has a variety of RAID solutions, ranging from specialized distributions, such as FreeNAS, to hardware RAID, as well as OS-level solutions such as
	<code>graid</code> on BSD and <code>mdadm</code> on Linux. These systems all report a variety of state data and metrics through a wide array of interfaces. With the release of 7.2, <a href="/products/beats/metricbeat">Metricbeat</a> now reports expanded metrics on Linux software RAID. In this post, we’ll look at the software RAID metrics available in Linux, and how to ingest them with Metricbeat.
</p><h2>Linux software RAID metrics</h2><p>Linux software RAID devices, usually created using <code>mdadm</code>, expose a variety of metric interfaces. One of these is via <code>ioctl</code>, which uses the <code>GET_DISK_INFO</code> and <code>GET_ARRAY_INFO</code> request codes. Users don’t often interact with <code>ioctl</code>, but you can see the data they return using tools like <code>mdadm --status</code>:
</p><pre class="”prettyprint”">sudo mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Tue Jun  4 15:29:29 2019
        Raid Level : raid1
        Array Size : 101376 (99.00 MiB 103.81 MB)
     Used Dev Size : 101376 (99.00 MiB 103.81 MB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent
       Update Time : Tue Jun  4 15:29:31 2019
             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0
Consistency Policy : resync
              Name : ubuntu-bionic:0  (local to host ubuntu-bionic)
              UUID : 03183b64:f3ec0a8e:f0e1d1fc:ea0f2bba
            Events : 17
    Number   Major   Minor   RaidDevice State
       0       8       33        0      active sync   /dev/sdc1
       1       8       34        1      active sync   /dev/sdc2
</pre><p>Another is <code>/proc/mdstat</code>, which provides a human-readable procfs interface:
</p><pre class="”prettyprint”">Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdc2[1] sdc1[0]
      101376 blocks super 1.2 [2/2] [UU]
unused devices: &lt;none&gt;
</pre><p>The final interface, and the one that Metricbeat utilizes as of 7.2, is <code>/sys/block/mdX</code>:
</p><pre class="”prettyprint”">ls -l /sys/block/md0/
total 0
drwxr-xr-x  9 root root    0 Jun  4 15:56 ./
drwxr-xr-x 11 root root    0 Jun  4 15:23 ../
-r--r--r--  1 root root 4096 Jun  4 15:53 alignment_offset
lrwxrwxrwx  1 root root    0 Jun  4 15:53 bdi -&gt; ../../bdi/9:0/
-r--r--r--  1 root root 4096 Jun  4 15:53 capability
-r--r--r--  1 root root 4096 Jun  4 15:29 dev
-r--r--r--  1 root root 4096 Jun  4 15:53 discard_alignment
-r--r--r--  1 root root 4096 Jun  4 15:53 ext_range
-r--r--r--  1 root root 4096 Jun  4 15:53 hidden
drwxr-xr-x  2 root root    0 Jun  4 15:53 holders/
-r--r--r--  1 root root 4096 Jun  4 15:53 inflight
drwxr-xr-x  2 root root    0 Jun  4 15:53 integrity/
drwxr-xr-x  5 root root    0 Jun  4 15:56 md/
drwxr-xr-x  2 root root    0 Jun  4 15:53 power/
drwxr-xr-x  2 root root    0 Jun  4 15:53 queue/
-r--r--r--  1 root root 4096 Jun  4 15:53 range
-r--r--r--  1 root root 4096 Jun  4 15:53 removable
-r--r--r--  1 root root 4096 Jun  4 15:53 ro
-r--r--r--  1 root root 4096 Jun  4 15:53 size
drwxr-xr-x  2 root root    0 Jun  4 15:53 slaves/
-r--r--r--  1 root root 4096 Jun  4 15:53 stat
lrwxrwxrwx  1 root root    0 Jun  4 15:53 subsystem -&gt; ../../../../class/block/
drwxr-xr-x  2 root root    0 Jun  4 15:53 trace/
-rw-r--r--  1 root root 4096 Jun  4 15:29 uevent
</pre><p>This sysfs interface provides <a href="https://www.kernel.org/doc/html/v4.15/admin-guide/md.html#md-devices-in-sysfs">a number of metrics</a> based on the RAID level of the array. As of 7.2, Metricbeat only reads from a few of these files: <code>size</code>, <code>md/array_state</code>, <code>md/level</code>,<code> md/dev-X/state</code>,  <code>md/sync_completed</code>, and <code>md/sync_action</code>. These last two are only available if the RAID array is redundant; that is, a 1, 4, 5, 6, or 10 array.
</p><h2>Ingesting Linux software RAID data with Metricbeat</h2><p>If you’re already using the Metricbeat system module, enable the RAID metricset. By default, the configuration is present, but commented out in <code>modules.d/system.yml</code>. Uncomment the lines to get started:
</p><pre class="”prettyprint”">#- module: system
#  period: 5m
#  metricsets:
#    - raid
#  raid.mount_point: '/'
</pre><p>If you’re running Metricbeat inside of Docker, <a href="/guide/en/beats/metricbeat/7.2/running-on-docker.html#monitoring-host">configure Metricbeat to monitor the host machine</a>, and set the <code>hostfs</code> variable in <code>system.yml</code> to reflect the sysfs mount point.
</p><p>After you configure the metricset, start Metricbeat and look for events from the RAID metricset:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blta3d61732b9ae329c/5cffd97ae646a764172fe094/image2.png" data-sys-asset-uid="blta3d61732b9ae329c" alt="JSON result of looking for events from the RAID metricset">
</p><p>Metricbeat normalizes disk states into four categories: <code>active</code>, <code>total</code>, <code>spare</code>, and <code>failed</code>. The <code>states</code> object reports the raw states from the operating system. This structure allows us to keep metrics consistent as we add more platforms to the metricset. The <code>blocks</code> object will show the state of any sync action in the array, or the total block size if no action is taking place.
</p><h2>Example: Finding failed disks in Kibana</h2><p>Once you’re ingesting RAID data into Elasticsearch, it’s easy to find RAID arrays with failed disks or other problems using the Discover app.
</p><p>Just create a new filter rule using the system.raid.disks.failed field, and filter for values other than 0:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt78efbedad1f38d3f/5cffd99b6a96af7017dfe419/image3.png" data-sys-asset-uid="blt78efbedad1f38d3f" alt="Creating a new filter rule in the Kibana Discover app">
</p><p>We can now see that the md1 array has a failed disk that should be addressed:
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blte6ea2f0b8137b596/5cffd9b8e646a764172fe09c/image1.png" data-sys-asset-uid="blte6ea2f0b8137b596" alt="Kibana Discover app showing that the md1 array has a failed disk">
</p><h2>Wrapping up</h2><p>And it’s that simple to start monitoring your Linux software RAIDs with Metricbeat and the Elastic Stack. You can try it out in a 14-day free trial of the <a href="/cloud/">Elasticsearch Service</a>, or <a href="/downloads">download</a> and run it locally today. If you run into problems, be sure to reach out on our <a href="https://discuss.elastic.co/c/beats">Discuss forum</a>.
</p>

kibana-filter-export-raid.jpg

linux-raid-metrics-thumb.jpg

Monitoring Linux software RAID metrics with Metricbeat and Elasticsearch

Monitoring Linux software RAID metrics with Metricbeat

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Blog Footer

Alex Kristiansen

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content.

You'll also receive an email with related content.

You'll also receive an email with related content

Load More

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

Elasticsearch Platform

ELK Stack

Elastic Cloud

Observability

Security

Search

By industry

By solution

Customer spotlight

Developers

Connect

Learn

Help

See what's happening at Elastic

Author

Articles by Alex Kristiansen

Monitoring Linux software RAID metrics with Metricbeat

Follow us

About us

Join us

Press

Partners

Trust & Security

Investor relations

EXCELLENCE AWARDS