Hot Threads APIedit

Warning

This functionality is experimental and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but experimental features are not subject to the support SLA of official GA features.

The hot threads API gets the current hot threads for Logstash. A hot thread is a Java thread that has high CPU usage and executes for a longer than normal period of time.

GET /_node/hot_threads

The output is a JSON document that contains a breakdown of the top hot threads for Logstash.

Example response:

{
  "hot_threads": {
    "time": "2016-09-19T10:44:13-07:00",
    "busiest_threads": 3,
    "threads": [
      {
        "name": "LogStash::Runner",
        "percent_of_cpu_time": 0.17,
        "state": "timed_waiting",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "java.lang.Thread.join(Thread.java:1253)",
          "org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
          "org.jruby.RubyThread.join(RubyThread.java:697)",
          "org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
          "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)",
          "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)",
          "org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)"
        ]
      },
      {
        "name": "Ruby-0-Thread-17",
        "percent_of_cpu_time": 0.11,
        "state": "timed_waiting",
        "path": "/Users/username/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:471",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "org.jruby.RubyThread.sleep(RubyThread.java:1002)",
          "org.jruby.RubyKernel.sleep(RubyKernel.java:803)",
          "org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)",
          "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)",
          "rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)",
          "rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)",
          "org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)"
        ]
      },
      {
        "name": "[main]-pipeline-manager",
        "percent_of_cpu_time": 0.04,
        "state": "timed_waiting",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "java.lang.Thread.join(Thread.java:1253)",
          "org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
          "org.jruby.RubyThread.join(RubyThread.java:697)",
          "org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)",
          "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)",
          "org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)"
        ]
      }
    ]
  }
}

The parameters allowed are:

threads

The number of hot threads to return. The default is 3.

human

If true, returns plain text instead of JSON format. The default is false.

ignore_idle_threads

If true, does not return idle threads. The default is true.

You can use the ?human parameter to return the document in a human-readable format.

GET /_node/hot_threads?human=true

Example of a human-readable response:

::: {}
Hot threads at 2016-07-26T18:46:18-07:00, busiestThreads=3:
================================================================================
 0.15 % of cpu usage by timed_waiting thread named 'LogStash::Runner'
        java.lang.Object.wait(Native Method)
        java.lang.Thread.join(Thread.java:1253)
        org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
        org.jruby.RubyThread.join(RubyThread.java:697)
        org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
        org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
        org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
        org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
        org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
 --------------------------------------------------------------------------------
 0.11 % of cpu usage by timed_waiting thread named 'Ruby-0-Thread-17'
 /Users/username/BuildTesting/logstash-5.0.0logstash-core/lib/logstash/pipeline.rb:471
        java.lang.Object.wait(Native Method)
        org.jruby.RubyThread.sleep(RubyThread.java:1002)
        org.jruby.RubyKernel.sleep(RubyKernel.java:803)
        org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)
        org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
        org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
        rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)
        rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)
        org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)
 --------------------------------------------------------------------------------
 0.04 % of cpu usage by timed_waiting thread named '[main]-pipeline-manager'
        java.lang.Object.wait(Native Method)
        java.lang.Thread.join(Thread.java:1253)
        org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
        org.jruby.RubyThread.join(RubyThread.java:697)
        org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
        org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
        org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)
        org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)
        org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)

See Common Options for a list of options that can be applied to all Logstash monitoring APIs.