Santa has arrived a few days early in Elastic Stack universe. Say heya to 7.0.0 alpha2.
But before we continue with the news, we do want to remind you that this is still an alpha. So, we recommend that you do not run this in production. There is no guarantee that 7.0.0-alpha1 versions will be compatible with other preview releases, or even with 7.0.0 GA.
Without further ado, we present to you the goodness of 7.0.0 Alpha2. Take it for a spin, and become an Elastic Pioneer. Learn about the Elastic Pioneer Program
A New Era for Cluster Coordination in Elasticsearch
Since the beginning, we focused on making Elasticsearch easy to scale and resilient to catastrophic failures. To support these requirements, we created a pluggable cluster coordination system, with the default implementation known as Zen Discovery. Zen Discovery was meant to be effortless, and give our users peace of mind (as the name implies). The meteoric rise in Elasticsearch usage has taught us a great deal. For instance, Zen’s minimum_master_nodes setting was often misconfigured, which put clusters at a greater risk of split brains and losing data. Maintaining this setting across large and dynamically resizing clusters was also difficult.
In Elasticsearch 7.0, we have completely rethought and rebuilt the cluster coordination layer. The new implementation gives safe sub-second master election times, where Zen may have taken several seconds to elect a new master, valuable time for a mission critical deployment. With the minimum_master_nodes setting removed, growing and shrinking clusters becomes safer and easier, and leaves much less room to mis-configure the system. Most importantly, the new cluster coordination layer gives us strong building blocks for the future of Elasticsearch, ensuring we can build functionality for even more advanced use cases to come.
Note that this new of cluster coordination layer requires an additional bootstrapping step when creating a new cluster for the first time. See the docs for more details.
Filebeat now Supports NetFlow
NetFlow is a network protocol developed by Cisco for collecting and monitoring network traffic. It contains information about connections traversing the network device, and includes source IP addresses and ports, destination IP addresses and ports, types of service, VLANs, and other information that can be encoded into frame and protocol headers. With Netflow data, network operators can go beyond monitoring simply the volume of data crossing their networks. They can understand where the traffic originated, where it is going, and what services or applications it is part of. A large number of routers and switch vendors support exporting NetFlow packets via UDP.
The new NetFlow input for Filebeat can be used to receive these Netflow and IPFIX records over UDP. It supports NetFlow v1, v5, v6, v7, v8, v9 and IPFIX. For Netflow versions older than 9, the fields are automatically mapped to v9 fields.
Auditbeat System Module
The new system module in Auditbeat collects data about the operating system, processes, sockets and users existing on a particular host. The Auditbeat system module generally sends two kind of events: state events, which contain the current state of the system (running processes, established sockets, etc), and diff events that tell one when a process was started/stopped, when a connection was established/dropped, when the a user logged in, etc. This is a convenient data model for Security Analytics use cases and results in a small disk footprint.
Try It Out
Happy downloading. Go forth, and explore.
Elasticsearch download | Release Notes
Kibana download | Release Notes
Logstash download | Release Notes
Beats downloads | Release Notes
ES-Hadoop download | Release Notes
APM Server download | Release Notes