Today, we announce new versions of the entire Elastic Stack, including a tighter integration of Shield with Kibana and an updated version of ES-Hadoop. Detailed blogs for product releases are available in the releases category of the blog. And yes, the blog has categories -- you know, for searchability.
Winlogbeat: A new beat for Windows Event logs
You like Filebeat, but you want to also send Windows Event logs to Elasticseach? There’s a Beat for that! It’s called Winlogbeat, and you can download it from here.
Winlogbeat supports both the new and old styles of the Windows event log APIs, meaning that you can use it on any Windows version starting with XP. Just like Filebeat, Winlogbeat has a registry file that tracks which events were acknowledged by Elasticsearch or Logstash, so you don’t lose events in case of restarts, network partitions, or Elasticsearch/Logstash unavailability.
Winlogbeat comes with a sample Kibana dashboard that you can use as a starting point for your customized dashboard. You can easily load the sample dashboards by using the load command described in the guide.
A commonly requested feature for Filebeat is to be able to merge related log lines into a single event. Think of the way most applications dump their exceptions into logs. Wouldn’t it be nice to have a single event per exception? This was already possible by using the Logstash multiline codec, but for many users, it’s more convenient to configure multiline in the same file where they configure the file paths.
The following example shows how to configure Filebeat to handle a multiline message where the first line of the message begins with a bracket ( [ ).
multiline: pattern: ^\[ negate: true match: after
Filebeat takes all the lines that do not start with [ and combines them with the previous line that does. For example, you could use this configuration to join the following lines of a multiline message into a single event:
[beat-logstash-some-name-832-2015.11.28] IndexNotFoundException[no such index] org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.resolve(IndexNameExpressionResolver.java:566) org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:133) org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:77) org.elasticsearch.action.admin.indices.delete.TransportDeleteIndexAction.checkBlock(TransportDeleteIndexAction.java:75
For more details about multiline, please check the docs.
It’s now possible to very efficiently filter lines out in Filebeat by using regular expressions. So if you were, for example, using Logstash to drop all debug messages, you can save a lot of bandwidth and CPU power by doing the filtering at the source.
You can configure Filebeat to drop all the log lines that match a certain regular expression or to include only the log lines that match a specific regular expression.
To export any lines that start with ERR or WARN:
include_lines: [“^ERR”, “^WARN”]
To drop any debug lines that start with DBG:
It’s also possible to ignore files based on the file names. Filebeat handles this efficiently by simply not opening the matching files.
To ignore all the files that have a gz extension:
Beats dashboards for windows users
Windows users are now able to load the default Kibana dashboards by using the load.ps1 script. It has similar options as the load.sh script for Unix systems, so you can use -url for passing the Elasticsearch URL or -user to authenticate with Elasticsearch by username and password.
Send us feedback!
A big Thank You to everyone who contributed code, reported issues, created a new Beat or just tried the Beats. Start your experience with Beats 1.1.0 and let us know what you think on discuss, Twitter, or open an issue on GitHub.