Machine Learning Anomaly Scoring and Elasticsearch - How it Works | Elastic Blog
工程

机器学习异常评分和 Elasticsearch —— 背后的原理

Elastic 使 ML

IP

:

  • 300
  • 300%
  • IP IP

1e-308 线

  • IP

0-100 UI

UI 75 100 50 75 25 50 0 25

enter image description here

90

API 5 farequote_count

GET /_xpack/ml/anomaly_detectors/farequote_count/results/records?human
{
  "sort": "record_score",
  "desc": true,
  "start": "2016-02-09T16:15:00.000Z",
  "end" :"2016-02-09T16:20:00.000Z"
}

{
  "count": 1,
  "records": [
    {
      "job_id": "farequote_count",
      "result_type": "record",
      "probability": 1.75744e-11,
      "record_score": 90.6954,
      "initial_record_score": 85.0643,
      "bucket_span": 300,
      "detector_index": 0,
      "is_interim": false,
      "timestamp_string": "2016-02-09T16:15:00.000Z",
      "timestamp": 1455034500000,
      "function": "count",
      "function_description": "count",
      "typical": [
        59.9827
      ],
      "actual": [
        179
      ]
    }
  ]
}

5 buckspanrecordscore90.6954 100 1.75744e-11 5 179 60

UI 1.75744 e-11 0 100

UI 3 便

ML

访访

bucket_time使

API

  • 航空公司 / API 计数
  • 航空公司 / API 平均(响应时间)

enter image description here

AAL 97 184 线 97 AAL

API

GET _xpack/ml/anomaly_detectors/farequote_count_and_responsetime_by_airline/results/influencers?human
{
  "start": "2016-02-09T16:15:00.000Z",
  "end" :"2016-02-09T16:20:00.000Z"
}

{
  "count": 2,
  "influencers": [
    {
      "job_id": "farequote_count_and_responsetime_by_airline",
      "result_type": "influencer",
      "influencer_field_name": "airline",
      "influencer_field_value": "AAL",
      "airline": "AAL",
      "influencer_score": 97.1547,
      "initial_influencer_score": 98.5096,
      "probability": 6.56622e-40,
      "bucket_span": 300,
      "is_interim": false,
      "timestamp_string": "2016-02-09T16:15:00.000Z",
      "timestamp": 1455034500000
    },
    {
      "job_id": "farequote_count_and_responsetime_by_airline",
      "result_type": "influencer",
      "influencer_field_name": "airline",
      "influencer_field_value": "AWE",
      "airline": "AWE",
      "influencer_score": 0,
      "initial_influencer_score": 0,
      "probability": 0.0499957,
      "bucket_span": 300,
      "is_interim": false,
      "timestamp_string": "2016-02-09T16:15:00.000Z",
      "timestamp": 1455034500000
    }
  ]
}

AAL 97.1547 UI 97 6.56622 e-40

98.5096 initialinformaterscore 97.1547/ AWE 0

influencer_score API 访

bucket_span

  • 使 byfields / partitionfields

使

  • 技术 航空公司 /
  • 平均(响应时间) 航空公司 /

enter image description here

线 UI bucket_spanUI

90 98 95

API

GET _xpack/ml/anomaly_detectors/farequote_count_and_responsetime_by_airline/results/buckets?human
{
  "start": "2016-02-09T16:15:00.000Z",
  "end" :"2016-02-09T16:20:00.000Z"
}

:

{
  "count": 1,
  "buckets": [
    {
      "job_id": "farequote_count_and_responsetime_by_airline",
      "timestamp_string": "2016-02-09T16:15:00.000Z",
      "timestamp": 1455034500000,
      "anomaly_score": 90.7,
      "bucket_span": 300,
      "initial_anomaly_score": 85.08,
      "event_count": 179,
      "is_interim": false,
      "bucket_influencers": [
        {
          "job_id": "farequote_count_and_responsetime_by_airline",
          "result_type": "bucket_influencer",
          "influencer_field_name": "airline",
          "initial_anomaly_score": 85.08,
          "anomaly_score": 90.7,
          "raw_anomaly_score": 37.3875,
          "probability": 6.92338e-39,
          "timestamp_string": "2016-02-09T16:15:00.000Z",
          "timestamp": 1455034500000,
          "bucket_span": 300,
          "is_interim": false
        },
        {
          "job_id": "farequote_count_and_responsetime_by_airline",
          "result_type": "bucket_influencer",
          "influencer_field_name": "bucket_time",
          "initial_anomaly_score": 85.08,
          "anomaly_score": 90.7,
          "raw_anomaly_score": 37.3875,
          "probability": 6.92338e-39,
          "timestamp_string": "2016-02-09T16:15:00.000Z",
          "timestamp": 1455034500000,
          "bucket_span": 300,
          "is_interim": false
        }
      ],
      "processing_time_ms": 17,
      "result_type": "bucket"
    }
  ]
}

  • 异常分数 - 90.7
  • initial_anomaly_score - anomaly_score anomaly_score UI initialanomalyscore
  • bucket_informater - influencerfieldname influencerfieldnamebucket_time API

使

使 influencerscore使recordscore

使 1 使 record_score 使

:

Elasticsearch v5.5 Elasticsearch