이 블로그에서는 명명된 엔티티 인식(NER)을 예로 들어 Elasticsearch에서 자연어 처리(Natural Language Processing, NLP)를 위한 딥 러닝 모델을 사용하여 시작하고 실행하는 프로세스를 설명합니다.

Elasticsearch에 NER 모델 배포

NER 모델 테스팅

유추 수집 파이프라인에 NER 모델 추가

유추를 위한 텍스트 필드 선택

nlp-getting-started-part2-les-miserables-entities-tag-cloud.png

배포 조정

더 자세히 알아보고 사용해 보기

illustration-a-new-state-of-find-campaign-image-608x440-2x.png

How to deploy NLP: Named entity recognition (NER) example

NLP를 배포하는 방법: 명명된 엔티티 인식(NER) 예제

이 블로그에서는 HuggingFace와 PyTorch를 사용하여 Elasticsearch에서 자연어 처리(NLP)를 위한 딥 러닝 모델을 시작하고 실행하는 프로세스를 설명합니다. 

blog-thumb-community-discussion.png

How to deploy natural language processing (NLP): Getting started

자연어 처리(NLP)를 배포하는 방법: 시작하기

이 블로그에서는 정서 분석을 예제 작업으로 삼아, Elasticsearch에서 자연어 처리(NLP)를 위한 딥 러닝 모델을 사용하여 시작 및 실행하는 프로세스를 설명합니다.

Elasticsearch에 정서 분석 모델 배포

Yelp 리뷰 분석

nlp-getting-started-part3-inference-count.png

얼마나 많은 리뷰가 부정적인지 시각화

nlp-getting-started-part3-sentiment-analysis-model.png

직접 사용해 보기

illustration-industries-retail.png

How to deploy NLP: Sentiment Analysis Example

NLP를 배포하는 방법: 정서 분석 예제

Read about the updates and bug fixes that have been included in this release

blog-banner-release-patch.png

blog-thumb-release-patch.png

Elastic Stack 7.14.1 released

Elastic Stack 7.11.2 has been released. Read about the updates and bug fixes that have been included.

Elastic Stack 7.11.2 released

The Elasticsearch inference pipeline aggregation lets you apply new inference models on data that's already been indexed. Learn how to use it to apply a new model to existing data and then visualize the results with the Kibana Vega plugin.

Elastic 7.6 introduced the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.6/inference-processor.html" target="_self">inference processor</a> for performing inference on documents as they are ingested through an ingest pipeline. Ingest pipelines are incredibly powerful and flexible but they are designed to work at ingest. So what happens if your data is already ingested?Introducing the new Elasticsearch <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.9/search-aggregations-pipeline-inference-bucket-aggregation.html" target="_self">inference pipeline aggregation</a>, which lets you apply new inference models on data that's already been indexed. With this new aggregation type, you can use the machine learning inference at search in an aggregation and get the results instantly — in real time with the latest data. Now you can always look forward to new models without worrying about reindexing your data in Elasticsearch!This blog will show you how to use the inference pipeline aggregation to apply a new model to existing data (for our example, we'll use customer service churn data), and will then show you how to dashboard the inference results with the <a href="https://www.elastic.co/guide/en/kibana/7.9/vega-graph.html" target="_self">Kibana Vega plugin</a>. If you'd like to try it yourself, you can spin up a <a href="/elasticsearch/service" target="_self">free trial of Elastic Cloud</a> or enable a <a href="https://www.elastic.co/guide/en/kibana/7.9/managing-licenses.html" target="_self">free trial of machine learning</a> on a <a href="/start" target="_self">local install</a>.<h2>Inference pipeline aggregations</h2>Inference is the process of making predictions against a model trained by machine learning algorithms. The Elastic Stack supports two types of models: regression models, which are trained to find relationships in the data and predict a numerical value such as house price, and classification models, which predict which category or class the data point falls into.&nbsp;The input to a model can be numerical values, categorical (text/keyword field), or even IP addresses. A pipeline aggregation is an aggregation that takes the output of other aggregations as its input. These might be numerical values, keywords, or even IP addresses — and now you can see the start of a beautiful friendship.&nbsp;<h2>Example: Applying a new customer churn model</h2>The data set we'll be using was first presented in the <a href="/webinars/introduction-to-supervised-machine-learning-in-elastic" target="_self">Introduction to supervised machine learning in Elastic</a> webinar. In that webinar, we used fictional telecom call record data (from <a href="https://www.openml.org/d/40701" target="_self">OpenML</a> and <a href="https://www.kaggle.com/becksddf/churn-in-telecoms-dataset" target="_self">Kaggle</a>) to predict which customers will churn given their history. A subset of the customer records have been marked up with a target churn field, making them suitable for training a model with supervised learning. Experimentation has shown features such as the total call charges and the number of customer service calls are particularly significant predictors of churn.&nbsp;We have the trained model from the webinar that tells us which customers are at risk of churning — now it is time to use it. In this blog, I'm going to show you how to execute that same model at search (not ingest) so you can view the inference predictions instantly on the latest data.Top Tip: When sharing a model between clusters, use the for_export parameter in the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.9/get-inference.html#ml-get-inference-query-params" target="_self">Get trained model</a> request to return the model in a format that can be directly uploaded to a different cluster.<h3>Step 1: Ingest customer churn data</h3>Download the data set and install&nbsp;the&nbsp;trained model to get started with this example:<ol><li>Download <a href="https://storage.googleapis.com/elastic-ml-public/supervised_ml_webinar_20200225/calls.csv" target="_self">calls.csv</a> and <a href="https://storage.googleapis.com/elastic-ml-public/supervised_ml_webinar_20200225/customers.csv" target="_self">customers.csv</a>.</li><li>Import the CSVs using the <a href="https://www.elastic.co/guide/en/kibana/7.9/connect-to-elasticsearch.html" target="_self">Data Visualizer</a> in Kibana, creating the calls and customer indices. If you don't have a Kibana instance handy, spin up a <a href="/elasticsearch/service" target="_self">free trial of Elastic Cloud</a>.</li><li>Download <a href="https://storage.googleapis.com/elastic-ml-public/supervised_ml_webinar_20200225/telco_churn_model.json" target="_self">telco_churn_model.json</a>.</li><li>Upload the churn model to your cluster with this curl command, changing user:password and localhost:9200 as appropriate.</li></ol><pre class="prettyprint">curl -u user:password -XPUT -H "Content-Type: application/json" "http://localhost:9200/_ml/inference/telco_churn" -d @telco_churn_model.json </pre><h3>Step 2: Using the inference aggregation</h3>The customer call data you just ingested is normalized and split across two indices — customers and calls. Phone numbers are unique to customers and occur in both indices; a composite terms aggregation on the phone_number field is used to create the customer entity and the features are constructed via subaggregations. The inference aggregation is a parent pipeline aggregation that operates on each bucket or entity.&nbsp;Run the inference aggregation below from within the <a href="https://www.elastic.co/guide/en/kibana/7.9/console-kibana.html" target="_self">Console</a>:<pre class="prettyprint">GET calls,customers/_search { "size": 0, "aggs": { "phone_number": { "composite": { "size": 100, "sources": [ { "phone_number": { "terms": { "field": "phone_number" } } } ] }, "aggs": { "call_charges": { "sum": { "field": "call_charges" } }, "call_duration": { "sum": { "field": "call_duration" } }, "call_count": { "value_count": { "field": "dialled_number" } }, "customer_service_calls": { "sum": { "field": "customer_service_calls" } }, "number_vmail_messages": { "sum": { "field": "number_vmail_messages" } }, "account_length": { "scripted_metric": { "init_script": "state.account_length = 0", "map_script": "state.account_length = params._source.account_length", "combine_script": "return state.account_length", "reduce_script": "for (d in states) if (d != null) return d" } }, "international_plan": { "scripted_metric": { "init_script": "state.international_plan = ''", "map_script": "state.international_plan = params._source.international_plan", "combine_script": "return state.international_plan", "reduce_script": "for (d in states) if (d != null) return d" } }, "voice_mail_plan": { "scripted_metric": { "init_script": "state.voice_mail_plan = ''", "map_script": "state.voice_mail_plan = params._source.voice_mail_plan", "combine_script": "return state.voice_mail_plan", "reduce_script": "for (d in states) if (d != null) return d" } }, "state": { "scripted_metric": { "init_script": "state.state = ''", "map_script": "state.state = params._source.state", "combine_script": "return state.state", "reduce_script": "for (d in states) if (d != null) return d" } }, "churn_classification": { "inference": { "model_id": "telco_churn", "inference_config": { "classification": { "prediction_field_type": "number" } }, "buckets_path": { "account_length": "account_length.value", "call_charges": "call_charges.value", "call_count": "call_count.value", "call_duration": "call_duration.value", "customer_service_calls": "customer_service_calls.value", "international_plan": "international_plan.value", "number_vmail_messages": "number_vmail_messages.value", "state": "state.value" } } } } } } } </pre>Granted, this looks complicated because each feature expected by the model is computed with a subaggregation. But let’s focus on the inference configuration, which is relatively simple.<pre class="prettyprint">"inference": { "model_id": "telco_churn", "inference_config": { "classification": { "prediction_field_type": "number" } }, "buckets_path": { "account_length": "account_length.value", "call_charges": "call_charges.value", "call_count": "call_count.value", "call_duration": "call_duration.value", "customer_service_calls": "customer_service_calls.value", "international_plan": "international_plan.value", "number_vmail_messages": "number_vmail_messages.value", "state": "state.value" } } </pre>The required fields are model_id — in this case the id of the previously trained model I was given — and buckets_path, a map detailing the inputs using the standard pipeline aggs bucket path <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline.html#buckets-path-syntax" target="_self">syntax</a>. The inference_config section specifies that the output should be a number rather than a string, which will prove useful in a moment.The output of the inference aggregation with the other aggs elided looks like this:<pre class="prettyprint">"aggregations" : { "phone_number" : { "meta" : { }, "after_key" : { "phone_number" : "408-337-7163" }, "buckets" : [ { "key" : { "phone_number" : "408-327-6764" }, ... "churn_classification" : { "value" : 0, "prediction_probability" : 0.9748441804026897, "prediction_score" : 0.533111462991059 } }, ... } } } </pre>value: 0 indicates the customer will not churn and the prediction probability tells us the model is confident of the result.We can see the predictions but the customers we are really interested in are those who the model predicts will churn. To find those fickle users we can filter with a <a href="/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-bucket-selector-aggregation.html" target="_self">bucket_selector</a> aggregation:&nbsp;<pre class="prettyprint">"will_churn_filter": { "bucket_selector": { "buckets_path": { "will_churn": "churn_classification&gt;value" }, "script": "params.will_churn &gt; 0" } } </pre>And now we only see the churners. The script requires the output of the aggregation is a numerical value, which is why we set "prediction_field_type": "number" in the inference configuration.<h3>Step 3: Visualizing the results in Kibana</h3>This is great, but the results format is a lot to parse if you’re not a computer! How can I get a quick overview such as a count of those who will churn against those who will not? Preferably a visual one? Stand up the versatile <a href="https://www.elastic.co/guide/en/kibana/7.9/vega-graph.html" target="_self">Kibana Vega plugin</a>, which can both transform data and visualize it.The data source is the same query as above and a Vega transform is used to group the classification results by predicted class and produce a count of each type. First, a lookup is used to give meaningful names to the numerical predicted class value.&nbsp;<pre class="prettyprint">"transform": [ { "lookup": "churn_classification.value", "from": { "data": { "values": [ {"category": 0, "classification_class": "Won't Churn"}, {"category": 1, "classification_class": "Will Churn"} ] }, "key": "category", "fields": ["classification_class"] } }, { "aggregate": [ { "op": "count", "as": "class_count" } ], "groupby": ["classification_class"] } ], </pre>Then draw a pie chart of the inference results.<pre class="prettyprint">"mark": "arc", "encoding": { "theta": {"field": "class_count", "type": "quantitative"}, "color": {"field": "classification_class", "type": "nominal", "legend": {"title": null }} } </pre>And we can save this as a simple dashboard.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt181ba30b9f40f772/5f75aa5ce69fa40ef318111e/prediction_dashboard.png" data-sys-asset-uid="blt181ba30b9f40f772" alt="Kibana dashboard of predicted category classes from the ml inference aggregation"/>All the search and vega configurations are stored in the <a href="https://gist.github.com/davidkyle/0bc9c84d60251c468a9e927c7228ba2d" target="_self">Gist</a> for this blog.<h2>Conclusion</h2>Everybody wants to use their historical data to make better business decisions, and this is a very practical, straightforward way to do just that. By using a machine learning model, the inference aggregation can quickly and easily be utilized to find customers on the verge of churning. Make predictions in real time and use the Kibana Vega visualization to dashboard the results.It is simple to get started — train a model using Elastic’s <a href="https://www.elastic.co/guide/en/kibana/7.9/xpack-ml-dfanalytics.html" target="_self">data frame analytics</a> or import an existing one with the <a href="https://github.com/elastic/eland#machine-learning-in-eland" target="_self">Eland</a> Python client. Try it for yourself today with a free trial of <a href="/cloud/" target="_self">Elastic Cloud</a>.

blog-banner-charts-monitor.png

blog-thumb-charts-monitor.png

Add flexibility to your data science with inference pipeline aggregations

Elastic Stack 7.6.2 has been released. Read about the updates and bug fixes that have been included.

Version 7.6.2 of the Elastic Stack was released today. We recommend you <a href="https://www.elastic.co/downloads">upgrade to this latest version</a>.
The 7.6.2 patch contains fixes and small enhancements for the stack. A notable bug fix in security is:
<ul>
	<li><a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-7009">Privilege escalation flaw</a>: an attacker who is able to generate an API key can perform a series of steps that result in an API key being generated with elevated privileges.</li>
</ul>For a full list of changes for each product, please refer to the release notes:
7.6.2 release notes
<ul>
	<li><a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.6/release-notes-7.6.2.html">Elasticsearch</a></li>
	<li><a href="https://www.elastic.co/guide/en/kibana/7.6/release-notes-7.6.2.html">Kibana</a></li>
	<li><a href="https://www.elastic.co/guide/en/logstash/7.6/logstash-7-6-2.html">Logstash</a></li>
	<li><a href="https://www.elastic.co/guide/en/beats/libbeat/7.6/release-notes-7.6.2.html">Beats</a></li>
</ul>

Elastic Stack 7.6.2 released

pexels-photo-712786.jpeg

adult-blur-bokeh-712786.jpg

Filtering Input Data to Refine Machine Learning Jobs

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

David Kyle

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

제출하면 귀하가 <a href="/kr/legal/elastic-cloud-account-terms" target="_self">Elastic 서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 연락처로 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">선택한 커뮤니케이션</a>을 수신하는 데 동의한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

제출하면 귀하가 Elastic <a href="/kr/legal/serverless-preview-terms" target="_self">서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 세부 정보를 사용하여 Elastic이 관련 제품 및 서비스에 대해 귀하에게 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">연락할 수 있음</a>을 이해한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

제출하면 <a href="/kr/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud 서비스 약관</a>에 동의하시게 됩니다. 개인 데이터는 <a href="/kr/legal/privacy-statement">Elastic의 개인정보 취급방침</a>에 따라 처리됩니다.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

제출하면 귀하가 Elastic <a href="/kr/legal/terms-of-use" target="_self">서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 세부 정보를 사용하여 Elastic이 관련 제품 및 서비스에 대해 귀하에게 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">연락할 수 있음</a>을 이해한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.

Explore similar demos

Overview

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

이 페이지의 콘텐츠는 선택하신 언어로 제공되지 않습니다. Elastic은 다양한 언어로 콘텐츠를 제공하기 위해 최선을 다하고 있습니다.조금만 더 기다려주세요!

Author

Articles by

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

내 표준 시간대로 웨비나 시작 시간 확인하기

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

참여가 어려우신가요? 등록하시면 녹화 자료를 보내드리겠습니다. 이메일로 관련 콘텐츠도 받아보실 수 있습니다.

Small image for

On-demand webinar

Featured webinar

Register to Attend

Share on Facebook

Reset all

Upcoming webinar

View more posts

Print

Hosted by

Sign In to Attend

View next

Follow us on Facebook

Share

Highlights

Read less

You'll also receive an email with related content

이메일을 통해 관련 내용을 보내드리겠습니다.

Thank you for registering. We will send you a confirmation email soon.

등록해 주셔서 감사합니다. 곧 확인 이메일을 보내드리겠습니다.

Author

Articles by David Kyle

Elastic

NLP를 배포하는 방법: 명명된 엔티티 인식(NER) 예제

자연어 처리(NLP)를 배포하는 방법: 시작하기

NLP를 배포하는 방법: 정서 분석 예제

Elastic Stack 7.14.1 released

Elastic Stack 7.11.2 released

Add flexibility to your data science with inference pipeline aggregations

Elastic Stack 7.6.2 released

Filtering Input Data to Refine Machine Learning Jobs

Sign up for Elastic Cloud free trial