Cross-cluster search, ingest node, rollover API, shrink API, field collapsing, unified highlighter . . . there's lots to love in Elasticsearch these days. Get up to speed on all things 5.x and see how 6.x will address pain points around scale, upgrading, recovery, and sparse data and disk usage.
Walk through all things ingest for Logstash 5.x, from dead letter and persistent queues to the Grok Debugger and new monitoring APIs. Then get caught up on new lightweight data shipper additions like Heartbeat and Metricbeat, as well as new modules that simplify the getting started process.
Car2go is an always-on business offering mobility service with cars to customers living in urban areas. Customers and cars constitute an IoT service generating data which must be processed and analyzed in real-time. E.g. vehicle connectivity and condition, position data, reservation and payment, registration and validation. Elasticsearch was introduced to all development teams as an offering, and as a result high quality data analysis can be generated based on systems inside status to all parts of the organization. Using DevOps methods each team is able to implement, modify and visualize data effectively. This gives a fast understanding of capacity, errors and business opportunities in real-time.
Data drives our modern world but most data is never seen and yet its implications can be wide ranging. Using the Elastic Stack and its data analytics techniques, Smart Tracing works like an x-ray to reveal what sits behind modern networks based on network data. It uses the hidden data which network equipment and devices use to communicate with each other to create new insights into performance issues, security threats and network faults.
To be an independent cloud provider and enforce additional security guidelines, Volkswagen developed their internal central logging and monitoring solution. The solution is based on Microservices implemented in Java as a log endpoint, Kafka for the processing pipeline, Elasticsearch for log storage and Kibana for visualization. In this talk we’ll explain the rationale around the decision to build our own service, the architecture and describe our experiences with the platform.