本篇博文以命名实体识别 (NER) 为例，介绍了在 Elasticsearch 中使用深度学习模型启动和运行自然语言处理 (NLP) 的过程。

部署 NER 模型到 Elasticsearch 中

测试 NER 模型

添加 NER 模型到 Ingest 推理管道

选择用于推理的文本字段

nlp-getting-started-part2-les-miserables-entities-tag-cloud.png

调整部署

学无止境，勤于尝试

illustration-a-new-state-of-find-campaign-image-608x440-2x.png

How to deploy NLP: Named entity recognition (NER) example

如何部署自然语言处理 (NLP)：命名实体识别 (NER) 示例

这篇博文以情感分析为例，介绍了在 Elasticsearch 中使用深度学习模型启动和运行自然语言处理的过程。

向 Elasticsearch 部署情感分析模型

分析 Yelp 评价

nlp-getting-started-part3-inference-count.png

对消极评价的数量进行可视化

nlp-getting-started-part3-sentiment-analysis-model.png

立即试用

illustration-industries-retail.png

How to deploy NLP: Sentiment Analysis Example

如何部署 NLP：情感分析示例

本篇博文介绍了使用 Hugging Face 和 PyTorch 在 Elasticsearch 中启动并运行自然语言处理 (NLP) 深度学习模型的过程。

blog-thumb-community-discussion.png

How to deploy natural language processing (NLP): Getting started

如何部署自然语言处理 (NLP)：开始使用

Read about the updates and bug fixes that have been included in this release

blog-banner-release-patch.png

blog-thumb-release-patch.png

Elastic Stack 7.14.1 released

Elastic Stack 7.11.2 has been released. Read about the updates and bug fixes that have been included.

Elastic Stack 7.11.2 released

The Elasticsearch inference pipeline aggregation lets you apply new inference models on data that's already been indexed. Learn how to use it to apply a new model to existing data and then visualize the results with the Kibana Vega plugin.

Elastic 7.6 introduced the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.6/inference-processor.html" target="_self">inference processor</a> for performing inference on documents as they are ingested through an ingest pipeline. Ingest pipelines are incredibly powerful and flexible but they are designed to work at ingest. So what happens if your data is already ingested?Introducing the new Elasticsearch <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.9/search-aggregations-pipeline-inference-bucket-aggregation.html" target="_self">inference pipeline aggregation</a>, which lets you apply new inference models on data that's already been indexed. With this new aggregation type, you can use the machine learning inference at search in an aggregation and get the results instantly — in real time with the latest data. Now you can always look forward to new models without worrying about reindexing your data in Elasticsearch!This blog will show you how to use the inference pipeline aggregation to apply a new model to existing data (for our example, we'll use customer service churn data), and will then show you how to dashboard the inference results with the <a href="https://www.elastic.co/guide/en/kibana/7.9/vega-graph.html" target="_self">Kibana Vega plugin</a>. If you'd like to try it yourself, you can spin up a <a href="/elasticsearch/service" target="_self">free trial of Elastic Cloud</a> or enable a <a href="https://www.elastic.co/guide/en/kibana/7.9/managing-licenses.html" target="_self">free trial of machine learning</a> on a <a href="/start" target="_self">local install</a>.<h2>Inference pipeline aggregations</h2>Inference is the process of making predictions against a model trained by machine learning algorithms. The Elastic Stack supports two types of models: regression models, which are trained to find relationships in the data and predict a numerical value such as house price, and classification models, which predict which category or class the data point falls into.&nbsp;The input to a model can be numerical values, categorical (text/keyword field), or even IP addresses. A pipeline aggregation is an aggregation that takes the output of other aggregations as its input. These might be numerical values, keywords, or even IP addresses — and now you can see the start of a beautiful friendship.&nbsp;<h2>Example: Applying a new customer churn model</h2>The data set we'll be using was first presented in the <a href="/webinars/introduction-to-supervised-machine-learning-in-elastic" target="_self">Introduction to supervised machine learning in Elastic</a> webinar. In that webinar, we used fictional telecom call record data (from <a href="https://www.openml.org/d/40701" target="_self">OpenML</a> and <a href="https://www.kaggle.com/becksddf/churn-in-telecoms-dataset" target="_self">Kaggle</a>) to predict which customers will churn given their history. A subset of the customer records have been marked up with a target churn field, making them suitable for training a model with supervised learning. Experimentation has shown features such as the total call charges and the number of customer service calls are particularly significant predictors of churn.&nbsp;We have the trained model from the webinar that tells us which customers are at risk of churning — now it is time to use it. In this blog, I'm going to show you how to execute that same model at search (not ingest) so you can view the inference predictions instantly on the latest data.Top Tip: When sharing a model between clusters, use the for_export parameter in the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.9/get-inference.html#ml-get-inference-query-params" target="_self">Get trained model</a> request to return the model in a format that can be directly uploaded to a different cluster.<h3>Step 1: Ingest customer churn data</h3>Download the data set and install&nbsp;the&nbsp;trained model to get started with this example:<ol><li>Download <a href="https://storage.googleapis.com/elastic-ml-public/supervised_ml_webinar_20200225/calls.csv" target="_self">calls.csv</a> and <a href="https://storage.googleapis.com/elastic-ml-public/supervised_ml_webinar_20200225/customers.csv" target="_self">customers.csv</a>.</li><li>Import the CSVs using the <a href="https://www.elastic.co/guide/en/kibana/7.9/connect-to-elasticsearch.html" target="_self">Data Visualizer</a> in Kibana, creating the calls and customer indices. If you don't have a Kibana instance handy, spin up a <a href="/elasticsearch/service" target="_self">free trial of Elastic Cloud</a>.</li><li>Download <a href="https://storage.googleapis.com/elastic-ml-public/supervised_ml_webinar_20200225/telco_churn_model.json" target="_self">telco_churn_model.json</a>.</li><li>Upload the churn model to your cluster with this curl command, changing user:password and localhost:9200 as appropriate.</li></ol><pre class="prettyprint">curl -u user:password -XPUT -H "Content-Type: application/json" "http://localhost:9200/_ml/inference/telco_churn" -d @telco_churn_model.json </pre><h3>Step 2: Using the inference aggregation</h3>The customer call data you just ingested is normalized and split across two indices — customers and calls. Phone numbers are unique to customers and occur in both indices; a composite terms aggregation on the phone_number field is used to create the customer entity and the features are constructed via subaggregations. The inference aggregation is a parent pipeline aggregation that operates on each bucket or entity.&nbsp;Run the inference aggregation below from within the <a href="https://www.elastic.co/guide/en/kibana/7.9/console-kibana.html" target="_self">Console</a>:<pre class="prettyprint">GET calls,customers/_search { "size": 0, "aggs": { "phone_number": { "composite": { "size": 100, "sources": [ { "phone_number": { "terms": { "field": "phone_number" } } } ] }, "aggs": { "call_charges": { "sum": { "field": "call_charges" } }, "call_duration": { "sum": { "field": "call_duration" } }, "call_count": { "value_count": { "field": "dialled_number" } }, "customer_service_calls": { "sum": { "field": "customer_service_calls" } }, "number_vmail_messages": { "sum": { "field": "number_vmail_messages" } }, "account_length": { "scripted_metric": { "init_script": "state.account_length = 0", "map_script": "state.account_length = params._source.account_length", "combine_script": "return state.account_length", "reduce_script": "for (d in states) if (d != null) return d" } }, "international_plan": { "scripted_metric": { "init_script": "state.international_plan = ''", "map_script": "state.international_plan = params._source.international_plan", "combine_script": "return state.international_plan", "reduce_script": "for (d in states) if (d != null) return d" } }, "voice_mail_plan": { "scripted_metric": { "init_script": "state.voice_mail_plan = ''", "map_script": "state.voice_mail_plan = params._source.voice_mail_plan", "combine_script": "return state.voice_mail_plan", "reduce_script": "for (d in states) if (d != null) return d" } }, "state": { "scripted_metric": { "init_script": "state.state = ''", "map_script": "state.state = params._source.state", "combine_script": "return state.state", "reduce_script": "for (d in states) if (d != null) return d" } }, "churn_classification": { "inference": { "model_id": "telco_churn", "inference_config": { "classification": { "prediction_field_type": "number" } }, "buckets_path": { "account_length": "account_length.value", "call_charges": "call_charges.value", "call_count": "call_count.value", "call_duration": "call_duration.value", "customer_service_calls": "customer_service_calls.value", "international_plan": "international_plan.value", "number_vmail_messages": "number_vmail_messages.value", "state": "state.value" } } } } } } } </pre>Granted, this looks complicated because each feature expected by the model is computed with a subaggregation. But let’s focus on the inference configuration, which is relatively simple.<pre class="prettyprint">"inference": { "model_id": "telco_churn", "inference_config": { "classification": { "prediction_field_type": "number" } }, "buckets_path": { "account_length": "account_length.value", "call_charges": "call_charges.value", "call_count": "call_count.value", "call_duration": "call_duration.value", "customer_service_calls": "customer_service_calls.value", "international_plan": "international_plan.value", "number_vmail_messages": "number_vmail_messages.value", "state": "state.value" } } </pre>The required fields are model_id — in this case the id of the previously trained model I was given — and buckets_path, a map detailing the inputs using the standard pipeline aggs bucket path <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline.html#buckets-path-syntax" target="_self">syntax</a>. The inference_config section specifies that the output should be a number rather than a string, which will prove useful in a moment.The output of the inference aggregation with the other aggs elided looks like this:<pre class="prettyprint">"aggregations" : { "phone_number" : { "meta" : { }, "after_key" : { "phone_number" : "408-337-7163" }, "buckets" : [ { "key" : { "phone_number" : "408-327-6764" }, ... "churn_classification" : { "value" : 0, "prediction_probability" : 0.9748441804026897, "prediction_score" : 0.533111462991059 } }, ... } } } </pre>value: 0 indicates the customer will not churn and the prediction probability tells us the model is confident of the result.We can see the predictions but the customers we are really interested in are those who the model predicts will churn. To find those fickle users we can filter with a <a href="/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-bucket-selector-aggregation.html" target="_self">bucket_selector</a> aggregation:&nbsp;<pre class="prettyprint">"will_churn_filter": { "bucket_selector": { "buckets_path": { "will_churn": "churn_classification&gt;value" }, "script": "params.will_churn &gt; 0" } } </pre>And now we only see the churners. The script requires the output of the aggregation is a numerical value, which is why we set "prediction_field_type": "number" in the inference configuration.<h3>Step 3: Visualizing the results in Kibana</h3>This is great, but the results format is a lot to parse if you’re not a computer! How can I get a quick overview such as a count of those who will churn against those who will not? Preferably a visual one? Stand up the versatile <a href="https://www.elastic.co/guide/en/kibana/7.9/vega-graph.html" target="_self">Kibana Vega plugin</a>, which can both transform data and visualize it.The data source is the same query as above and a Vega transform is used to group the classification results by predicted class and produce a count of each type. First, a lookup is used to give meaningful names to the numerical predicted class value.&nbsp;<pre class="prettyprint">"transform": [ { "lookup": "churn_classification.value", "from": { "data": { "values": [ {"category": 0, "classification_class": "Won't Churn"}, {"category": 1, "classification_class": "Will Churn"} ] }, "key": "category", "fields": ["classification_class"] } }, { "aggregate": [ { "op": "count", "as": "class_count" } ], "groupby": ["classification_class"] } ], </pre>Then draw a pie chart of the inference results.<pre class="prettyprint">"mark": "arc", "encoding": { "theta": {"field": "class_count", "type": "quantitative"}, "color": {"field": "classification_class", "type": "nominal", "legend": {"title": null }} } </pre>And we can save this as a simple dashboard.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt181ba30b9f40f772/5f75aa5ce69fa40ef318111e/prediction_dashboard.png" data-sys-asset-uid="blt181ba30b9f40f772" alt="Kibana dashboard of predicted category classes from the ml inference aggregation"/>All the search and vega configurations are stored in the <a href="https://gist.github.com/davidkyle/0bc9c84d60251c468a9e927c7228ba2d" target="_self">Gist</a> for this blog.<h2>Conclusion</h2>Everybody wants to use their historical data to make better business decisions, and this is a very practical, straightforward way to do just that. By using a machine learning model, the inference aggregation can quickly and easily be utilized to find customers on the verge of churning. Make predictions in real time and use the Kibana Vega visualization to dashboard the results.It is simple to get started — train a model using Elastic’s <a href="https://www.elastic.co/guide/en/kibana/7.9/xpack-ml-dfanalytics.html" target="_self">data frame analytics</a> or import an existing one with the <a href="https://github.com/elastic/eland#machine-learning-in-eland" target="_self">Eland</a> Python client. Try it for yourself today with a free trial of <a href="/cloud/" target="_self">Elastic Cloud</a>.

blog-banner-charts-monitor.png

blog-thumb-charts-monitor.png

Add flexibility to your data science with inference pipeline aggregations

Elastic Stack 7.6.2 has been released. Read about the updates and bug fixes that have been included.

Version 7.6.2 of the Elastic Stack was released today. We recommend you <a href="https://www.elastic.co/downloads">upgrade to this latest version</a>.
The 7.6.2 patch contains fixes and small enhancements for the stack. A notable bug fix in security is:
<ul>
	<li><a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-7009">Privilege escalation flaw</a>: an attacker who is able to generate an API key can perform a series of steps that result in an API key being generated with elevated privileges.</li>
</ul>For a full list of changes for each product, please refer to the release notes:
7.6.2 release notes
<ul>
	<li><a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.6/release-notes-7.6.2.html">Elasticsearch</a></li>
	<li><a href="https://www.elastic.co/guide/en/kibana/7.6/release-notes-7.6.2.html">Kibana</a></li>
	<li><a href="https://www.elastic.co/guide/en/logstash/7.6/logstash-7-6-2.html">Logstash</a></li>
	<li><a href="https://www.elastic.co/guide/en/beats/libbeat/7.6/release-notes-7.6.2.html">Beats</a></li>
</ul>

Elastic Stack 7.6.2 released

pexels-photo-712786.jpeg

adult-blur-bokeh-712786.jpg

Filtering Input Data to Refine Machine Learning Jobs

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

David Kyle

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

提交即表示您确认您已阅读并同意我们的<a href="/cn/legal/elastic-cloud-account-terms" target="_self">服务条款</a>，并且您同意通过您提供的上述联系方式接收<a href="/cn/legal/privacy-statement#how-we-use-the-information" target="_self">所选的通信</a>。如需了解更多信息，或想随时取消订阅，请参阅 <a href="/cn/legal/privacy-statement/" target="_self">Elastic 的隐私声明</a>。

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

提交即表示您确认您已阅读并同意我们的<a href="/cn/legal/serverless-preview-terms" target="_self">服务条款</a>，并且 Elastic 可以使用您提供的上述详细信息与您联系，以向您介绍我们相关的<a href="/cn/legal/privacy-statement#how-we-use-the-information" target="_self">产品和服务</a>。如需了解更多信息，或想随时取消订阅，请参阅 <a href="/cn/legal/privacy-statement/" target="_self">Elastic 的隐私声明</a>。

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

提交即表示您同意 <a href="/cn/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud 服务条款</a>。我们将会按照 <a href="/cn/legal/privacy-statement">Elastic 的隐私声明</a>处理您的个人数据。

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

提交即表示您确认您已阅读并同意我们的<a href="/cn/legal/terms-of-use" target="_self">服务条款</a>，并且 Elastic 可以使用您提供的上述详细信息与您联系，以向您介绍我们相关的<a href="/cn/legal/privacy-statement#how-we-use-the-information" target="_self">产品和服务</a>。如需了解更多信息，或想随时取消订阅，请参阅 <a href="/cn/legal/privacy-statement/" target="_self">Elastic 的隐私声明</a>。

Explore similar demos

Overview

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

本页内容尚不支持所选语言。Elastic 正在不断努力，以实现对多种语言内容的支持。感谢您在此期间给予的耐心与陪伴！

Author

Articles by

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

查看本次网络研讨会在我的时区的开始时间

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

无法实时参加？注册后我们会给您发送录像。您还将收到相关内容的电子邮件。

Author

Articles by David Kyle

Elastic

如何部署自然语言处理 (NLP)：命名实体识别 (NER) 示例

如何部署 NLP：情感分析示例

如何部署自然语言处理 (NLP)：开始使用

Elastic Stack 7.14.1 released

Elastic Stack 7.11.2 released

Add flexibility to your data science with inference pipeline aggregations

Elastic Stack 7.6.2 released

Filtering Input Data to Refine Machine Learning Jobs

Sign up for Elastic Cloud free trial