Maltiverse Integration


Version	1.12.0 (View all)
Managed integration release status	GA
Subscription level What's this?	Basic
Developed by What's this?	Partner
Ingestion method(s)	API
Minimum Kibana version(s)	9.1.0 8.19.2

Important - Managed integrations on Elastic Cloud Hosted

You can use this integration as an Elastic Managed integration on Elastic Cloud Hosted deployments running this version or later.

Maltiverse is a threat intelligence platform. It works as a broker for Threat intelligence sources that are aggregated from more than a hundred different Public, Private and Community sources. Once the data is ingested, the IoC Scoring Algorithm applies a qualitative classification to the IoC that changes. Finally this data can be queried in a Threat Intelligence feed that can be delivered to your Firewalls, SOAR, SIEM, EDR or any other technology.

This integration fetches Maltiverse Threat Intelligence feeds and add them into Elastic Threat Intelligence. It supports hostname, hash, ipv4 and url indicators.

In order to download feed you need to register and generate an API key on you profile page.

Agentless Enabled Integration

Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to Agentless integrations and the Agentless integrations FAQ. Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.

IoCs Expiration

Since we want to retain only valuable information and avoid duplicated data, the Maltiverse Elastic integration forces the indicators to rotate into a custom index called: logs-ti_maltiverse_latest.indicator. Please, refer to this index in order to set alerts and so on.

How it works

This is possible thanks to a transform rule installed along with the integration. The transform rule parses the data_stream content that is pulled from Maltiverse and only adds new indicators.

Both, the data_stream and the latest index have applied expiration through ILM and a retention policy in the transform respectively. See Data retention.

Data retention

Threat indicators are re-collected across polling intervals, and the latest transform keeps the active, deduplicated view in its destination index. The source data streams therefore hold repeated copies of the same indicators.

The package bounds the growth of these source data streams with a retention that depends on the deployment type:

Data stream	Self-managed and Elastic Cloud Hosted (ILM policy)	Serverless (data stream lifecycle)
`logs-ti_maltiverse.indicator-*`	`logs-ti_maltiverse.indicator-default_policy`: roll over after 1d, delete 2d after rollover	delete 3d after rollover

On self-managed and Elastic Cloud Hosted deployments the ILM policy applies. The data stream lifecycle shipped with the package is not used there. ILM counts the delete age from the rollover of a backing index, so a document can remain for up to the rollover age plus the delete age. On Serverless, ILM is not available and the data stream lifecycle applies instead. It also works per backing index: Elasticsearch rolls the write index over automatically on age, size, or document count, and deletes a backing index once the retention has passed since it rolled over. A document therefore stays for the retention plus up to one rollover interval. The rollover age is derived from the retention and is an implementation detail that Elasticsearch may change. Where the package installs a transform, the transform's destination indices are not affected by either.

To keep data for a different period:

Self-managed and Elastic Cloud Hosted: edit the ILM policy in Kibana under Stack Management → Index Lifecycle Policies, or with PUT _ilm/policy/<policy name>. A package upgrade reinstalls the package's ILM policies, so check your change after upgrading.
Serverless: set the retention on the data stream, for example PUT _data_stream/logs-ti_maltiverse.indicator-default/_lifecycle with the body {"data_retention": "90d"}. Replace default with your namespace.

Logs

Indicator

Field	Description	Type
@timestamp	Event timestamp.	date
data_stream.dataset	Data stream dataset.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
event.dataset	Event dataset	constant_keyword
event.module	Event module	constant_keyword
input.type	Input type.	keyword
labels.is_ioc_transform_source	Indicates whether an IOC is in the raw source data stream, or the in latest destination index.	constant_keyword
maltiverse.address	registered address	keyword
maltiverse.address.address	Multi-field of `maltiverse.address`.	match_only_text
maltiverse.as_name	AS registered name	keyword
maltiverse.as_name.as_name	Multi-field of `maltiverse.as_name`.	match_only_text
maltiverse.asn_cidr	CIDR associated	keyword
maltiverse.asn_country_code	Country code asociated with ASN	keyword
maltiverse.asn_date	date when asn registered	date
maltiverse.asn_registry	ASN registry	keyword
maltiverse.blacklist.count	number of reports for the indicator	long
maltiverse.blacklist.description	what we saw	keyword
maltiverse.blacklist.description.description	Multi-field of `maltiverse.blacklist.description`.	match_only_text
maltiverse.blacklist.external_references		flattened
maltiverse.blacklist.first_seen	first sighting	date
maltiverse.blacklist.labels		keyword
maltiverse.blacklist.last_seen	last sighting	date
maltiverse.blacklist.source	reporter of the activity	keyword
maltiverse.cidr	CIDR associated	keyword
maltiverse.city	City	keyword
maltiverse.classification	Classification of the threat	keyword
maltiverse.country_code	Country code of the threat	keyword
maltiverse.creation_time	creation date	date
maltiverse.domain_consonants		long
maltiverse.domain_length		long
maltiverse.email	email address	keyword
maltiverse.entropy		double
maltiverse.feed	Origin of the IoC	keyword
maltiverse.hostname		keyword
maltiverse.ip_addr	IP address	ip
maltiverse.is_alive		boolean
maltiverse.is_cdn	boolean description tag	boolean
maltiverse.is_cnc	boolean description tag	boolean
maltiverse.is_distributing_malware	boolean description tag	boolean
maltiverse.is_hosting	boolean description tag	boolean
maltiverse.is_iot_threat	boolean description tag	boolean
maltiverse.is_known_attacker	boolean description tag	boolean
maltiverse.is_known_scanner	boolean description tag	boolean
maltiverse.is_mining_pool	boolean description tag	boolean
maltiverse.is_open_proxy	boolean description tag	boolean
maltiverse.is_phishing		boolean
maltiverse.is_sinkhole	boolean description tag	boolean
maltiverse.is_storing_phishing		boolean
maltiverse.is_tor_node	boolean description tag	boolean
maltiverse.is_vpn_node	boolean description tag	boolean
maltiverse.last_online_time		keyword
maltiverse.location	Longitude and latitude.	geo_point
maltiverse.modification_time	Last modification date	date
maltiverse.number_of_blacklisted_domains_resolving	Blacklisted domains resolving associated	long
maltiverse.number_of_domains_resolving	Domains resolving associated	long
maltiverse.number_of_offline_malicious_urls_allocated	URLs allocated	long
maltiverse.number_of_online_malicious_urls_allocated	URLs allocated	long
maltiverse.number_of_whitelisted_domains_resolving	Whitelisted domains resolving associated	long
maltiverse.postal_code		keyword
maltiverse.registrant_name	Registrant name	keyword
maltiverse.registrant_name.registrant_name	Multi-field of `maltiverse.registrant_name`.	match_only_text
maltiverse.resolved_ip		flattened
maltiverse.tag	Tags of the threat	keyword
maltiverse.type	Type of the threat	keyword
maltiverse.urlchecksum		keyword
threat.indicator.first_seen	The date and time when intelligence source first reported sighting this indicator.	date
threat.indicator.last_seen	The date and time when intelligence source last reported sighting this indicator.	date
threat.indicator.modified_at	The date and time when intelligence source last modified information for this indicator.	date

		{
    "@timestamp": "2022-11-05T05:37:57.000Z",
    "agent": {
        "ephemeral_id": "c371b9d1-ae14-4272-9d73-3ef7bf7e46f9",
        "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.13.0"
    },
    "data_stream": {
        "dataset": "ti_maltiverse.indicator",
        "namespace": "34244",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb",
        "snapshot": false,
        "version": "8.13.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "threat"
        ],
        "created": "2024-08-02T05:34:15.473Z",
        "dataset": "ti_maltiverse.indicator",
        "id": "NsHdp9tZZtzo6Kzlv6Z1TmPP47U=",
        "ingested": "2024-08-02T05:34:27Z",
        "kind": "enrichment",
        "original": "{\"blacklist\":{\"count\":1,\"description\":\"QakBot\",\"first_seen\":\"2022-11-03 06:23:53\",\"labels\":[\"malicious-activity\"],\"last_seen\":\"2022-11-05 05:37:57\",\"source\":\"ThreatFox Abuse.ch\"},\"classification\":\"malicious\",\"creation_time\":\"2022-11-03 06:23:53\",\"domain\":\"autooutletllc.com\",\"hostname\":\"autooutletllc.com\",\"is_alive\":false,\"is_cnc\":true,\"is_distributing_malware\":false,\"is_iot_threat\":false,\"is_phishing\":false,\"last_online_time\":\"2022-11-05 05:37:57\",\"modification_time\":\"2022-11-05 05:37:57\",\"tag\":[\"bb05\",\"iso\",\"qakbot\",\"qbot\",\"quakbot\",\"tr\",\"w19\",\"zip\",\"oakboat\",\"pinkslipbot\"],\"tld\":\"com\",\"type\":\"url\",\"url\":\"https://autooutletllc.com/spares.php\",\"urlchecksum\":\"4aa7a29969dc1dffa5cad5af6cb343b9a9b40ea9646fed619d4c8d6472629128\"}",
        "severity": 9,
        "type": [
            "indicator"
        ]
    },
    "input": {
        "type": "httpjson"
    },
    "maltiverse": {
        "blacklist": {
            "labels": [
                "malicious-activity"
            ]
        },
        "classification": "malicious",
        "creation_time": "2022-11-03T06:23:53.000Z",
        "feed": "test",
        "hostname": "autooutletllc.com",
        "is_alive": false,
        "is_cnc": true,
        "is_distributing_malware": false,
        "is_iot_threat": false,
        "is_phishing": false,
        "last_online_time": "2022-11-05T05:37:57.000Z",
        "modification_time": "2022-11-05T05:37:57.000Z",
        "type": "url",
        "urlchecksum": "4aa7a29969dc1dffa5cad5af6cb343b9a9b40ea9646fed619d4c8d6472629128"
    },
    "tags": [
        "preserve_original_event",
        "forwarded",
        "ti_maltiverse-indicator",
        "bb05",
        "iso",
        "qakbot",
        "qbot",
        "quakbot",
        "tr",
        "w19",
        "zip",
        "oakboat",
        "pinkslipbot"
    ],
    "threat": {
        "feed": {
            "reference": "https://maltiverse.com/feed/test"
        },
        "indicator": {
            "confidence": "High",
            "description": "QakBot",
            "first_seen": "2022-11-03T06:23:53.000Z",
            "last_seen": "2022-11-05T05:37:57.000Z",
            "marking": {
                "tlp": "WHITE"
            },
            "provider": "ThreatFox Abuse.ch",
            "reference": "https://maltiverse.com/url/4aa7a29969dc1dffa5cad5af6cb343b9a9b40ea9646fed619d4c8d6472629128",
            "sightings": 1,
            "type": "url",
            "url": {
                "full": "https://autooutletllc.com/spares.php",
                "registered_domain": "autooutletllc.com",
                "top_level_domain": "com"
            }
        }
    }
}
		
	

Changelog

Version	Details	Minimum Kibana version
1.12.0	Enhancement (View pull request) Document the retention applied to source data streams on stateful and Serverless deployments, and how to change it.	9.1.0 8.19.2
1.11.0	Enhancement (View pull request) Remove redundant ECS field definitions from the transform, relying on the `ecs@mappings` component template.	9.1.0 8.19.2
1.10.0	Enhancement (View pull request) Enable request trace log removal.	9.0.5 8.19.2
1.9.0	Enhancement (View pull request) Add tags to ingest pipeline processors and add `preserve_original_event` to pipeline-level `on_failure` handlers. Bug fix (View pull request) Remove a redundant no-op `rename` processor for `threat.indicator.geo.country_iso_code` in the `indicator` data stream.	9.0.5 8.19.2
1.8.2	Enhancement (View pull request) Set agentless deployment mode `release` field to `ga`.	9.0.5 8.19.2
1.8.1	Bug fix (View pull request) Fix transform destination to use keyword for data_stream.namespace, allowing multi-namespace deployments.	9.0.5 8.19.2
1.8.0	Enhancement (View pull request) Enable Agentless deployment.	9.0.5 8.19.2
1.7.0	Enhancement (View pull request) Use num_failure_retries instead of unattended mode for transform failure recovery.	9.0.0 8.13.0
1.6.0	Enhancement (View pull request) Allow transforms to run in unattended mode.	9.0.0 8.13.0
1.5.0	Enhancement (View pull request) Update Kibana constraint to support 9.0.0.	9.0.0 8.13.0
1.4.0	Enhancement (View pull request) Do not remove `event.original` in main ingest pipeline.	8.13.0
1.3.0	Enhancement (View pull request) Add "preserve_original_event" tag to documents with `event.kind` set to "pipeline_error".	8.13.0
1.2.5	Bug fix (View pull request) Use triple-brace Mustache templating when referencing variables in ingest pipelines.	8.13.0
1.2.4	Bug fix (View pull request) Use triple-brace Mustache templating when referencing variables in ingest pipelines.	8.13.0
1.2.3	Bug fix (View pull request) Fix labels.is_ioc_transform_source values	8.13.0
1.2.2	Bug fix (View pull request) Add missing fields in transform	8.13.0
1.2.1	Bug fix (View pull request) Fix ECS date mapping on threat fields.	8.13.0
1.2.0	Enhancement (View pull request) Update the kibana constraint to ^8.13.0. Modified the field definitions to remove ECS fields made redundant by the ecs@mappings component template.	8.13.0
1.1.1	Bug fix (View pull request) Add missing fields for detection rules.	8.12.0
1.1.0	Enhancement (View pull request) Set sensitive values as secret.	8.12.0
1.0.1	Enhancement (View pull request) Changed owners	8.8.0
1.0.0	Enhancement (View pull request) Release package as GA.	8.8.0
0.8.0	Enhancement (View pull request) Limit request tracer log count to five.	8.8.0
0.7.0	Enhancement (View pull request) ECS version updated to 8.11.0.	8.8.0
0.6.0	Enhancement (View pull request) Improve 'event.original' check to avoid errors if set.	8.8.0
0.5.0	Enhancement (View pull request) Set 'partner' owner type.	8.8.0
0.4.0	Bug fix (View pull request) Move non-ECS fields out of root.	8.8.0
0.3.0	Enhancement (View pull request) Add DLM policy. Add owner.type to package manifest. Update format_version to 3.0.0 Enhancement (View pull request) Add tags.yml file so that integration's dashboards and saved searches are tagged with "Security Solution" and displayed in the Security Solution UI.	8.8.0
0.2.1	Bug fix (View pull request) Remove dotted YAML keys.	—
0.2.0	Enhancement (View pull request) Add support for HTTP request trace logging.	8.8.0
0.1.0	Enhancement (View pull request) initial implementation	8.8.0