There is a distinction between inspiration and paraphrasing. It is possible to read a content, get inspired, and then explore the idea with your own words, even if you come to a similar conclusion.

While plagiarism has been a topic of discussion for a long time, the accelerated production and publication of content have kept it relevant and posed an ongoing challenge.

This challenge isn't limited to books, academic research, or judicial documents, where plagiarism checks are frequently conducted. It can also extend to newspapers and even social media.

With the abundance of information and easy access to publishing, how can plagiarism be effectively checked on a scalable level?

Universities, government entities, and companies employ diverse tools, but while a straightforward lexical search can effectively detect direct plagiarism, the primary challenge lies in identifying paraphrased content.

Plagiarism detection with Generative AI

A new challenge emerges with Generative AI. Is content generated by AI considered plagiarism when copied?

The OpenAI terms of use, for example, specify that OpenAI will not claim copyright over content generated by the API for users. In this case, individuals using their Generative AI can use the generated content as they prefer without citation.

However, the acceptance of using Generative AI to improve efficiency remains a topic of discussion.

In an effort to contribute to plagiarism detection, OpenAI developed a detection model but later acknowledged that its accuracy is not sufficiently high.

"We believe this is not high enough accuracy for standalone detection and needs to be paired with metadata-based approaches, human judgment, and public education to be more effective."

The challenge persists; however, with the availability of more tools, there are now increased options for detecting plagiarism, even in cases of paraphrased and AI content.

Detecting plagiarism with Elasticsearch

Recognizing this, in this blog we are exploring one more use case with Natural Language Processing (NLP) models and Vector Search, plagiarism detection, beyond metadata searches.

This is demonstrated with Python examples, where we utilize a dataset from SentenceTransformers containing NLP-related articles. We check the abstracts for plagiarism by performing 'semantic textual similarity' considering 'abstract' embeddings generated with a text embedding model previously imported into Elasticsearch. Additionally, to identify AI-generated content — AI plagiarism, an NLP model developed by OpenAI was also imported into Elasticsearch.

The following image illustrates the data flow:

During the ingest pipeline with an inference processor, the 'abstract' paragraph is mapped to a 768-dimensional vector, the 'abstract_vector.predicted_value'.

Mapping:

The similarity between vector representations is measured using a vector similarity metric, defined using the 'similarity' parameter.

Cosine is the default similarity metric, computed as '(1 + cosine(query, vector)) / 2'. Unless you need to preserve the original vectors and cannot normalize them in advance, the most efficient way to perform cosine similarity is to normalize all vectors to unit length. This helps avoid performing extra vector length computations during the search, instead use 'dot_product'.

In this same pipeline, another inference processor containing the text classification model detects whether the content is 'Real' probably written by humans, or 'Fake' probably written by AI, adding the 'openai-detector.predicted_value' to each document.

Ingest Pipeline:

At query time, the same text embedding model is also employed to generate the vector representation of the query 'model_text' in a 'query_vector_builder' object.

A k-nearest neighbor (kNN) search finds the k nearest vector to the query vector measured by the similarity metric.

The _score of each document is derived from the similarity, ensuring that a larger score corresponds to a higher ranking. This means that the document is more similar semantically. As a result, we are printing three possibilities: if score > 0.9, we are considering 'high similarity'; if < 0.7, 'low similarity’, otherwise, 'moderate similarity’. You have the flexibility to set different threshold values to determine what level of _score qualifies as plagiarism or not, based on your use case.

Additionally, text classification is performed to also check for AI-generated elements in the text query.

Query:

Output:

In this example, after utilizing one of the 'abstract' values from our dataset as the text query 'model_text', plagiarism was identified. The similarity score is 1.0, indicating a high level of similarity — direct plagiarism. The vectorized query and document were not recognized as AI-generated content, which was expected.

Query:

Output:

By updating the text query 'model_text' with an AI-generated text that conveys the same message while minimizing the repetition of similar words, the detected similarity was still high, but the score was 0.9302529 instead of 1.0 — paraphrase plagiarism. It was also expected that this query, which was generated by AI, would be detected.

Lastly, considering the text query 'model_text' as a text about Elasticsearch, which is not an abstract of one of these documents, the detected similarity was 0.68991005, indicating low similarity according to the considered threshold values.

Query:

Output:

Although plagiarism was accurately identified in the text query generated by AI, as well as in cases of paraphrasing and direct copied content, navigating the landscape of plagiarism detection involves acknowledging various aspects.

In the context of AI-generated content detection, we explored a model that makes a valuable contribution. However, it is crucial to recognize the inherent limitations in standalone detection, necessitating the incorporation of other methods to boost the accuracy.

The variability introduced by the choice of text embedding models is another consideration. Different models, trained with distinct datasets, result in varying levels of similarity, highlighting the importance of the text embeddings generated.

Lastly, in these examples, we used the document's abstract. However, plagiarism detection often involves large documents, making it essential to address the challenge of text length. It is common for the text to exceed a model's token limit, requiring segmentation into chunks before building embeddings. A practical approach to handling this involves utilizing nested structures with dense_vector.

Conclusion

In this blog, we discussed the challenges of detecting plagiarism, particularly in paraphrased and AI-generated content, and how semantic textual similarity and text classification can be used for this purpose.

By combining these methods, we provided an example of plagiarism detection where we successfully identified AI-generated content, direct and paraphrased plagiarism.

The primary goal was to establish a filtering system that simplifies detection but human assessment remains essential for validation.

If you are interested in learning more about semantic textual similarity and NLP, we encourage you to also check out these links:

• What is semantic search?

• What is natural language processing (NLP)?

• Lexical and Semantic Search with Elasticsearch

• Chunking Large Documents via Ingest pipelines plus nested vectors equals easy passage search

As technology advances, chatbots are used to handle more complex tasks — and quickly — while still providing a personalized experience for users. Natural language processing (NLP) enables chatbots to process the user's language, identifies the intent behind their message, and extracts relevant information from it. For example, Named Entity Recognition extracts key information in a text by classifying them into a set of categories. Sentiment Analysis identifies the emotional tone, and Question Answering the “answer” to a query. The goal of NLP is to enable algorithms to process human language and perform tasks that historically only humans were capable of, such as finding relevant passages among large amounts of text, summarizing text, and generating new, original content.

These advanced NLP capabilities are built upon a technology known as vector search. Elastic has native support for vector search, performing exact and approximate k-nearest neighbor (kNN) search, and for NLP, enabling the use of custom or third-party models directly in Elasticsearch.

In this blog post, we will explore how vector search and NLP work to enhance chatbot capabilities and demonstrate how Elasticsearch facilitates the process. Let's begin with a brief overview of vector search.

Vector search

Although humans can comprehend the meaning and context of written language, machines cannot do the same. This is where vectors come in. By converting text into vector representations (numerical representations of the meaning of the text), machines can overcome this limitation. Compared to a traditional search, instead of relying on keywords and lexical search based on frequencies, vectors enable the process of text data using operations defined for numerical values.

This allows vector search to locate data that shares similar concepts or contexts by using distances in the "embedding space" to represent similarity given a query vector. When the data is similar, the corresponding vectors will be alike.

Vector search is not only utilized in NLP applications, but it’s also used in various other domains where unstructured data is involved, including image and video processing.

In a chatbot flow, there can be several approaches to users' queries, and as a result, there are different ways to improve information retrieval for a better user experience. Since each alternative has its own set of advantages and possible disadvantages, it is essential to take into account the available data and resources, as well as the training time (when applicable) and expected accuracy. In the following section, we will cover these aspects for question-answering NLP models.

Question-answering

A question-answering (QA) model is a type of NLP model that is designed to answer questions asked in natural language. When users have questions that require inferring answers from multiple resources, without a pre-existing target answer available in the documents, generative QA models can be useful. However, these models can be computationally expensive and require large amounts of data for domain related training, which may make them less practical in some situations, even though this method can be particularly valuable to handle out-of-domain questions.

On the other hand, when users have questions on a specific topic, and the actual answer is present in the document, extractive QA models can be used. These models directly extract the answer from the source document, providing transparent and verifiable results, making them a more practical option for businesses or organizations that want to provide a simple and efficient way of answering questions.

The example below demonstrates the use of a pre-trained extractive QA model, available on Hugging Face and deployed into Elasticsearch, to extract answers from a given context:

Deploy trained models.

Add a model to an inference ingest pipeline.

There are various ways to handle user queries and retrieve information, and using multiple language models and data sources can be an effective alternative when dealing with unstructured data. To illustrate this, we have an example of the data processing of a chatbot employed to respond to queries with answers considering data extracted from selected documents.

Chatbot data processing: NLP and vector search

As shown above, the data processing for our chatbot can be divided into three parts:

• Vector processing: This part converts documents into vector representations.

• User input processing: This part extracts relevant information from the user query and performs semantic search and hybrid retrieval.

• Optimization: This part includes monitoring and is crucial for ensuring the chatbot's reliability, optimal performance, and great user experience.

Vector processing

For the processing part, the first step is to determine component parts of each document to then convert each element to a vector representation; these representations can be created for a wide range of data formats.

There are various methods that can be used to compute embeddings, including pre-trained models and libraries.

It's important to note that the effectiveness of search and retrieval on these representations depends on the existing data and the quality and relevance of the method used.

As the vectors are computed, they are stored in Elasticsearch with a dense_vector field type.

Chatbot user input processing

For the user part, after receiving a question, it's useful to extract all possible information from it before proceeding. This helps to understand the user's intention, and in this case, we are using a Named Entity Recognition model (NER) to assist with that. NER is the process of identifying and classifying named entities into predefined entity categories.

Although not a necessary step, by using structured data or the above or another NLP model result to categorize the user's query, we can restrict the kNN search using a filter. This helps to improve performance and accuracy by reducing the amount of data that needs to be processed.

Semantic search and hybrid retrieval

Since the prompt originates from user queries and the chatbot needs to process human language with its variability and ambiguity, semantic search is a great fit. In Elasticsearch, you can perform semantic search in a single step by passing the query string and the ID of the embedding model into a query_vector_builder object. This will vectorize the query and perform kNN search to retrieve top k matches that are closest in meaning to the query:

End-to-end example: How to deploy a text embedding model and use it for semantic search. Elasticsearch uses the Lucene implementation of the Okapi BM25, a sparse model , to rank text queries for relevance, while dense models are used for semantic search. To combine the strengths of both , vector matches and matches obtained from the text query, you can perform a hybrid retrieval :

Combining both sparse and dense models often yields the best results

Sparse models generally perform better on short queries and specific terminologies, while dense models leverage context and associations. If you want to learn more about how these methods compare and complement each other, here we benchmark BM25 against two dense models that have been specifically trained for retrieval.

The most relevant result can usually be the first answer given to the user, the_score is a number used to determine the relevance of the returned document.

Chatbot optimization

To help improve the user experience, performance, and reliability of your chatbot, in addition to applying hybrid scoring, you can incorporate the following approaches: Sentiment Analysis: To provide awareness of user comments and reactions as the dialog unfolds, you can incorporate a sentiment analysis model:

GPT's capabilities : As an alternative to enhance the overall experience, you can combine Elasticsearch's search relevance with OpenAI's GPT question-answering capabilities, utilizing the Chat Completion API to return to the user model-generated responses considering these top k documents as a context. Prompt: "answer this question <user_question> using only this document <top_search_result>"

Observability: Ensuring the performance of any chatbot is crucial, and monitoring is an essential component in achieving this. In addition to logs that capture chatbot interactions, it's important to track response time, latency, and other relevant chatbot metrics. By doing so, you can identify patterns, trends, and even detect anomalies.Elastic Observability tools enable you to collect and analyze this information.

Summary

This blog post covers what NLP and vector search are and delves into an example of a chatbot employed to respond to user queries by considering data extracted from the vector representation of documents.

As demonstrated, using NLP and vector search, chatbots are capable of performing complex tasks that go beyond structured, targeted data. This includes making recommendations and answering specific product or business-related queries using multiple data sources and formats as context, while also providing a personalized user experience.

Use cases range from providing customer service by assisting customers with their inquiries to helping developers with their queries, by providing step-by-step guidance, suggesting recommendations, or even automating tasks. Depending on the goal and existing data, other models and methods can also be utilized to achieve even better results and improve the overall user experience.

Here are some links on the topic that may be useful:

1. How to deploy natural language processing (NLP): Getting started

2. Overview of image similarity search in Elasticsearch

3. ChatGPT and Elasticsearch: OpenAI meets private data

4. Monitor OpenAI API and GPT models with OpenTelemetry and Elastic

5. 5 reasons IT leaders need vector search to improve search experiences

By incorporating NLP and native vector search in Elasticsearch, you can take advantage of its speed, scalability, and search capabilities to create highly efficient and effective chatbots capable of handling large amounts of data, whether structured or unstructured.

Ready to get started? Begin a free trial of Elastic Cloud.

In this blog post, we may have used or we may refer to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.

Elastic, Elasticsearch and associated marks are trademarks, logos or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.