Cassandra Integration

Version	1.16.0 (View all)
Compatible Kibana version(s)	8.13.0 or higher
Supported Serverless project types What’s this?	Security Observability
Subscription level What’s this?	Basic
Level of support What’s this?	Elastic

This integration periodically fetches metrics from Cassandra using jolokia agent. It can parse System logs.

Compatibility

edit

This integration has been tested against Cassandra version 3.11.11.

Steps to Setup Jolokia

edit

Prerequisites

edit

Java Development Kit (JDK) 1.8 or later
Apache Cassandra 3.x or 4.x (depending on user’s version)
Jolokia agent JAR file

Jolokia Setup

edit

Follow these steps to set up Jolokia for monitoring Apache Cassandra:

Download the Jolokia JVM Agent:

Visit the Jolokia official download page to obtain the latest version of the Jolokia JVM agent JAR file. Download the jolokia-jvm-<jolokia_version>-agent.jar file.
Copy the Jolokia Agent to Cassandra’s Library Directory:

Copy the downloaded jolokia-jvm-<jolokia_version>-agent.jar file to the Cassandra library directory on the server where Cassandra is installed.

For example:
```
cp jolokia-jvm-<jolokia_version>-agent.jar /path/to/cassandra/lib/
```
Replace /path/to/cassandra/lib/ with the actual path to Cassandra’s library directory.
Configure Cassandra to use the Jolokia Agent:

Open the cassandra-env.sh file, located in the Cassandra configuration directory, using a text editor, and add the following line at the bottom of the file:
```
JVM_OPTS="$JVM_OPTS -javaagent:/path/to/jolokia-jvm-<jolokia_version>-agent.jar=port=<jolokia_port>,host=0.0.0.0"
```
Replace /path/to/jolokia-jvm-<version>-agent.jar with the actual path to the Jolokia agent JAR file copied in Step 2. Save the changes and close the cassandra-env.sh file.
Restart Cassandra:

Restart the Apache Cassandra service to apply the changes made to the configuration.

Note:
- Restarting the Apache Cassandra service will temporarily disrupt database connectivity. Ensure that dependent services are designed to handle such interruptions gracefully.
- Immediately after a restart, Cassandra’s performance may be impacted due to cold caches and commit log replay. Allow some time for the system to stabilize.
- Before restarting Cassandra, ensure that no cluster maintenance tasks are in progress to prevent any unintended consequences.
- The exact steps will vary based on the installation type, the setup process might differ based on the specific deployment method or environment.
- Procedures for restarting Cassandra may vary based on user’s specific setup and configuration.

Verifying the setup

edit

After restarting Cassandra, user can verify that Jolokia is properly set up by accessing the Jolokia endpoint:

http://<cassandra-host>:<jolokia_port>/jolokia

Replace with the hostname or IP address of user’s Cassandra server.

If the setup is successful, user should see a JSON response containing information about the available Jolokia operations and the Cassandra instance.

User can now use Jolokia to monitor and manage Apache Cassandra cluster.

Troubleshooting

edit

If log.flags appears conflicted under the logs-* data view, this issue can be resolved by reindexing the Logs data stream.

Logs

edit

Cassandra system logs from cassandra.log files.

Example

An example event for log looks as following:

{
    "@timestamp": "2024-06-18T06:33:32.952Z",
    "agent": {
        "ephemeral_id": "b1e9fa09-5c73-45d9-b26f-184761635dd9",
        "id": "97400795-188c-4140-a1ee-0002078c785d",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "cassandra.log",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "97400795-188c-4140-a1ee-0002078c785d",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "database"
        ],
        "dataset": "cassandra.log",
        "ingested": "2024-06-18T06:34:02Z",
        "kind": "event",
        "module": "cassandra",
        "type": [
            "info"
        ]
    },
    "input": {
        "type": "log"
    },
    "log": {
        "file": {
            "path": "/tmp/service_logs/cassandra/system.log"
        },
        "level": "INFO",
        "offset": 0,
        "origin": {
            "file": {
                "line": 92,
                "name": "YamlConfigurationLoader.java"
            }
        }
    },
    "message": "Configuration location: file:/etc/cassandra/cassandra.yaml",
    "process": {
        "thread": {
            "name": "main"
        }
    },
    "tags": [
        "forwarded",
        "cassandra-systemlogs"
    ]
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields

Field	Description	Type
@timestamp	Event timestamp.	date
cassandra.log.meta	Log meta infos like java stack_trace.	keyword
data_stream.dataset	Data stream dataset.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
input.type	Type of Filebeat input.	keyword
log.flags	Flags for the log file.	keyword
log.offset	Offset of the entry in the log file.	long

Metrics

edit

Cassandra metrics using jolokia agent installed on cassandra.

Example

An example event for metrics looks as following:

{
    "@timestamp": "2024-06-18T06:35:01.265Z",
    "agent": {
        "ephemeral_id": "51e65675-8699-4d2e-8c14-ecde813096e9",
        "id": "97400795-188c-4140-a1ee-0002078c785d",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "8.13.0"
    },
    "cassandra": {
        "metrics": {
            "cache": {
                "key_cache": {
                    "capacity": 104857600,
                    "one_minute_hit_rate": 0,
                    "requests": {
                        "one_minute_rate": 12
                    }
                },
                "row_cache": {
                    "capacity": 0,
                    "requests": {
                        "one_minute_rate": 0
                    }
                }
            },
            "client": {
                "connected_native_clients": 0
            },
            "client_request": {
                "casread": {
                    "one_minute_rate": 0
                },
                "caswrite": {
                    "one_minute_rate": 0
                },
                "range_slice": {
                    "one_minute_rate": 0,
                    "total_latency": 0
                },
                "read": {
                    "count": 0,
                    "one_minute_rate": 0,
                    "timeouts": 0,
                    "timeoutsms": 0,
                    "total_latency": 0,
                    "unavailables": 0,
                    "unavailablesms": 0
                },
                "write": {
                    "count": 0,
                    "one_minute_rate": 0,
                    "timeouts": 0,
                    "timeoutsms": 0,
                    "total_latency": 0,
                    "unavailables": 0,
                    "unavailablesms": 0
                }
            },
            "column_family": {
                "total_disk_space_used": 72566
            },
            "compaction": {
                "completed": 44,
                "pending": 0
            },
            "dropped_message": {
                "batch_remove": 0,
                "batch_store": 0,
                "counter_mutation": 0,
                "hint": 0,
                "mutation": 0,
                "paged_range": 0,
                "range_slice": 0,
                "read": 0,
                "read_repair": 0,
                "request_response": 0,
                "trace": 0
            },
            "gc": {
                "concurrent_mark_sweep": {
                    "collection_count": 1,
                    "collection_time": 26
                },
                "par_new": {
                    "collection_count": 1,
                    "collection_time": 29
                }
            },
            "memory": {
                "heap_usage": {
                    "committed": 4054777856,
                    "init": 4158652416,
                    "max": 4054777856,
                    "used": 481894272
                },
                "other_usage": {
                    "committed": 62337024,
                    "init": 2555904,
                    "max": -1,
                    "used": 60729840
                }
            },
            "storage": {
                "exceptions": 0,
                "load": 72566,
                "total_hint_in_progress": 0,
                "total_hints": 0
            },
            "system": {
                "cluster": "Test Cluster",
                "data_center": "datacenter1",
                "live_nodes": "192.168.247.2",
                "rack": "rack1",
                "version": "3.11.11"
            },
            "table": {
                "all_memtables_heap_size": 4584,
                "all_memtables_off_heap_size": 0,
                "live_disk_space_used": 72566,
                "live_ss_table_count": 11
            },
            "task": {
                "complete": 55,
                "pending": 0,
                "total_commitlog_size": 67108864
            },
            "thread_pools": {
                "counter_mutation_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "mutation_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "read_repair_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "read_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "request_response_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                }
            }
        }
    },
    "data_stream": {
        "dataset": "cassandra.metrics",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "97400795-188c-4140-a1ee-0002078c785d",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "database"
        ],
        "created": "2024-06-18T06:35:01.265Z",
        "dataset": "cassandra.metrics",
        "duration": 110507236,
        "ingested": "2024-06-18T06:35:13Z",
        "kind": "event",
        "module": "cassandra",
        "type": [
            "info"
        ]
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "8259e024976a406e8a54cdbffeb84fec",
        "ip": [
            "192.168.245.7"
        ],
        "mac": [
            "02-42-C0-A8-F5-07"
        ],
        "name": "docker-fleet-agent",
        "os": {
            "codename": "focal",
            "family": "debian",
            "kernel": "3.10.0-1160.102.1.el7.x86_64",
            "name": "Ubuntu",
            "platform": "ubuntu",
            "type": "linux",
            "version": "20.04.6 LTS (Focal Fossa)"
        }
    },
    "metricset": {
        "name": "jmx",
        "period": 10000
    },
    "service": {
        "address": "http://elastic-package-service-cassandra-1:8778/jolokia/%3FignoreErrors=true&canonicalNaming=false",
        "type": "jolokia"
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

Exported fields

Field	Description	Type	Metric Type
@timestamp	Event timestamp.	date
agent.id	Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id.	keyword
cassandra.metrics.cache.key_cache.capacity		long	gauge
cassandra.metrics.cache.key_cache.one_minute_hit_rate		long	gauge
cassandra.metrics.cache.key_cache.requests.one_minute_rate		long	gauge
cassandra.metrics.cache.row_cache.capacity		long	gauge
cassandra.metrics.cache.row_cache.one_minute_hit_rate		long	gauge
cassandra.metrics.cache.row_cache.requests.one_minute_rate		long	gauge
cassandra.metrics.client.connected_native_clients		long	gauge
cassandra.metrics.client_request.casread.one_minute_rate		double	gauge
cassandra.metrics.client_request.caswrite.one_minute_rate		double	gauge
cassandra.metrics.client_request.range_slice.one_minute_rate		double	gauge
cassandra.metrics.client_request.range_slice.total_latency		double	counter
cassandra.metrics.client_request.read.count		long	counter
cassandra.metrics.client_request.read.one_minute_rate		double	gauge
cassandra.metrics.client_request.read.timeouts	Number of read timeouts encountered.	double	counter
cassandra.metrics.client_request.read.timeoutsms		double	gauge
cassandra.metrics.client_request.read.total_latency		double	counter
cassandra.metrics.client_request.read.unavailables	Number of read unavailables encountered.	double	counter
cassandra.metrics.client_request.read.unavailablesms		double	gauge
cassandra.metrics.client_request.write.count		long	counter
cassandra.metrics.client_request.write.one_minute_rate		double	gauge
cassandra.metrics.client_request.write.timeouts		double	counter
cassandra.metrics.client_request.write.timeoutsms		double	gauge
cassandra.metrics.client_request.write.total_latency		double	counter
cassandra.metrics.client_request.write.unavailables		double	counter
cassandra.metrics.client_request.write.unavailablesms		double	gauge
cassandra.metrics.column_family.total_disk_space_used		long	gauge
cassandra.metrics.compaction.completed	compaction completed tasks.	long	gauge
cassandra.metrics.compaction.pending	compaction pending tasks.	long	gauge
cassandra.metrics.dropped_message.batch_remove		long	counter
cassandra.metrics.dropped_message.batch_store		long	counter
cassandra.metrics.dropped_message.counter_mutation		long	counter
cassandra.metrics.dropped_message.hint		long	counter
cassandra.metrics.dropped_message.mutation		long	counter
cassandra.metrics.dropped_message.paged_range		long	counter
cassandra.metrics.dropped_message.range_slice		long	counter
cassandra.metrics.dropped_message.read		long	counter
cassandra.metrics.dropped_message.read_repair		long	counter
cassandra.metrics.dropped_message.request_response		long	counter
cassandra.metrics.dropped_message.trace		long	counter
cassandra.metrics.gc.concurrent_mark_sweep.collection_count	Total number of CMS collections that have occurred.	long	gauge
cassandra.metrics.gc.concurrent_mark_sweep.collection_time	Approximate accumulated CMS collection elapsed time in milliseconds.	long	gauge
cassandra.metrics.gc.par_new.collection_count	Total number of ParNew collections that have occurred.	long	gauge
cassandra.metrics.gc.par_new.collection_time	Approximate accumulated ParNew collection elapsed time in milliseconds.	long	gauge
cassandra.metrics.memory.heap_usage.committed	Committed heap memory usage.	long	gauge
cassandra.metrics.memory.heap_usage.init	Initial heap memory usage.	long	gauge
cassandra.metrics.memory.heap_usage.max	Max heap memory usage.	long	gauge
cassandra.metrics.memory.heap_usage.used	Used heap memory usage.	long	gauge
cassandra.metrics.memory.other_usage.committed	Committed non-heap memory usage.	long	gauge
cassandra.metrics.memory.other_usage.init	Initial non-heap memory usage.	long	gauge
cassandra.metrics.memory.other_usage.max	Max non-heap memory usage.	long	gauge
cassandra.metrics.memory.other_usage.used	Used non-heap memory usage.	long	gauge
cassandra.metrics.storage.exceptions	The number of the total exceptions.	long	counter
cassandra.metrics.storage.load	Storage used for Cassandra data in bytes.	long	counter
cassandra.metrics.storage.total_hint_in_progress	The number of the total hits in progress.	long	counter
cassandra.metrics.storage.total_hints	The number of the total hits.	long	counter
cassandra.metrics.system.cluster		keyword
cassandra.metrics.system.data_center		keyword
cassandra.metrics.system.joining_nodes		keyword
cassandra.metrics.system.leaving_nodes		keyword
cassandra.metrics.system.live_nodes		keyword
cassandra.metrics.system.moving_nodes		keyword
cassandra.metrics.system.rack		keyword
cassandra.metrics.system.unreachable_nodes		keyword
cassandra.metrics.system.version		keyword
cassandra.metrics.table.all_memtables_heap_size		long	gauge
cassandra.metrics.table.all_memtables_off_heap_size		long	gauge
cassandra.metrics.table.live_disk_space_used		long	counter
cassandra.metrics.table.live_ss_table_count		long	gauge
cassandra.metrics.task.complete	completed tasks.	long	gauge
cassandra.metrics.task.pending	pending tasks.	long	gauge
cassandra.metrics.task.total_commitlog_size	total commitlog size of tasks.	long	gauge
cassandra.metrics.thread_pools.counter_mutation_stage.request.active		long	gauge
cassandra.metrics.thread_pools.counter_mutation_stage.request.pending		long	gauge
cassandra.metrics.thread_pools.mutation_stage.request.active		long	gauge
cassandra.metrics.thread_pools.mutation_stage.request.pending		long	gauge
cassandra.metrics.thread_pools.read_repair_stage.request.active		long	gauge
cassandra.metrics.thread_pools.read_repair_stage.request.pending		long	gauge
cassandra.metrics.thread_pools.read_stage.request.active		long	gauge
cassandra.metrics.thread_pools.read_stage.request.pending		long	gauge
cassandra.metrics.thread_pools.request_response_stage.request.active		long	gauge
cassandra.metrics.thread_pools.request_response_stage.request.pending		long	gauge
cloud.account.id	The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.	keyword
cloud.availability_zone	Availability zone in which this host, resource, or service is located.	keyword
cloud.instance.id	Instance ID of the host machine.	keyword
cloud.provider	Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.	keyword
cloud.region	Region in which this host, resource, or service is located.	keyword
container.id	Unique container id.	keyword
data_stream.dataset	Data stream dataset.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
host.name	Name of the host. It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.	keyword
service.address	Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).	keyword

Changelog

edit

Changelog

Version	Details	Kibana version(s)
1.16.0	Enhancement (View pull request) Allow @custom pipeline access to event.original without setting preserve_original_event.	8.13.0 or higher
1.15.0	Enhancement (View pull request) Add processor support for metrics data stream.	8.13.0 or higher
1.14.0	Enhancement (View pull request) ECS version updated to 8.11.0. Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.	8.13.0 or higher
1.13.0	Enhancement (View pull request) Add global filter on data_stream.dataset to improve performance.	8.12.0 or higher
1.12.2	Enhancement (View pull request) Add documentation for setting up Jolokia to monitor Apache Cassandra.	8.12.0 or higher
1.12.1	Enhancement (View pull request) Update README with reindexing steps.	8.12.0 or higher
1.12.0	Enhancement (View pull request) Enable secrets for sensitive fields. For more details, refer https://www.elastic.co/guide/en/fleet/current/agent-policy.html#agent-policy-secret-values	8.12.0 or higher
1.11.1	Bug fix (View pull request) Disable secrets for older stack versions due to errors.	8.8.0 or higher
1.11.0	Enhancement (View pull request) Enable secret for the sensitive fields, supported from 8.12.	8.8.0 or higher
1.10.1	Bug fix (View pull request) Update the link to the correct reindexing procedure.	8.8.0 or higher
1.10.0	Enhancement (View pull request) Update the package format_version to 3.0.0.	8.8.0 or higher
1.9.2	Bug fix (View pull request) Fix the type for `log.flags` field.	8.8.0 or higher
1.9.1	Bug fix (View pull request) Add null check and ignore_missing check to the rename processor	8.8.0 or higher
1.9.0	Enhancement (View pull request) Enable time series data streams for the metrics datasets. This dramatically reduces storage for metrics and is expected to progressively improve query performance. For more details, see https://www.elastic.co/guide/en/elasticsearch/reference/current/tsds.html.	8.8.0 or higher
1.8.1	Enhancement (View pull request) Add metric_type mapping for the fields of `metrics` datastream.	8.3.0 or higher
1.8.0	Enhancement (View pull request) Add dimension fields for metrics datastream for TSDB enablement.	8.3.0 or higher
1.7.0	Enhancement (View pull request) Rename ownership from obs-service-integrations to obs-infraobs-integrations	8.3.0 or higher
1.6.0	Enhancement (View pull request) Migrate `System Logs` dashboard visualizations to lens.	8.3.0 or higher
1.5.0	Enhancement (View pull request) Migrate `Overview` dashboard visualizations to lens.	8.3.0 or higher
1.4.1	Enhancement (View pull request) Added categories and/or subcategories.	7.15.0 or higher 8.0.0 or higher
1.4.0	Enhancement (View pull request) Update ECS version to 8.5.1	7.15.0 or higher 8.0.0 or higher
1.3.0	Enhancement (View pull request) Update cassandra package as per best practices.	7.15.0 or higher 8.0.0 or higher
1.2.3	Bug fix (View pull request) Fix dashboard issues.	7.15.0 or higher 8.0.0 or higher
1.2.2	Bug fix (View pull request) Fix typo in config template for ignoring host enrichment	—
1.2.1	Enhancement (View pull request) Add documentation for multi-fields	—
1.2.0	Enhancement (View pull request) Update to ECS 8.0	—
1.1.0	Enhancement (View pull request) Release cassandra package for v8.0.0	7.15.0 or higher 8.0.0 or higher
1.0.0	Enhancement (View pull request) GA Release	—
0.0.1	Enhancement (View pull request) Initial draft of the package	—

« Canva CEL Custom API input integration »