人工智能剽窃利用 Elasticsearch 检测剽窃行为
以下是如何使用 Elasticsearch 检查人工智能剽窃，重点是使用 NLP 模型和矢量搜索的用例。
剽窃可以是直接剽窃，即抄袭部分或全部内容；也可以是转述剽窃，即对作者的作品进行改写，改动一些词或短语。
灵感和意译是有区别的。阅读一篇内容，获得启发，然后用自己的语言来探讨这个想法，即使得出的结论相似，也是有可能的。
虽然剽窃一直是讨论的话题，但内容的加速生产和发布使其具有现实意义，并构成了持续的挑战。
这种挑战并不局限于书籍、学术研究或司法文件，在这些领域经常会进行抄袭检查。它还可以延伸到报纸甚至社交媒体。
随着信息的丰富和出版的便捷，如何在可扩展的层面上有效检查抄袭行为？
大学、政府机构和公司使用的工具多种多样，虽然直接的词汇搜索可以有效检测直接抄袭，但主要的挑战在于识别转述的内容。
利用生成式人工智能检测剽窃行为
生成式人工智能带来了新的挑战。人工智能生成的内容在复制时是否被视为抄袭？
例如，OpenAI的使用条款规定，OpenAI 不会对 API 为用户生成的内容主张版权。在这种情况下，使用生成式人工智能的个人可以随意使用生成的内容，而无需引用。
不过，是否接受使用生成式人工智能来提高效率仍是一个讨论话题。
为了对剽窃检测做出贡献，OpenAI 开发了一个检测模型，但后来承认其准确性不够高。
"我们认为，这对于独立检测来说精度还不够高，需要与基于元数据的方法、人工判断和公众教育相结合，才能更加有效。"
然而，随着更多工具的出现，现在有了更多检测抄袭的选择，即使是在转述和人工智能内容的情况下。
使用 Elasticsearch 检测剽窃行为
有鉴于此，在本博客中，我们将利用自然语言处理（NLP）模型和矢量搜索（plagiarism detection）来探索元数据搜索之外的另一个使用案例。
我们利用 SentenceTransformers 提供的 数据集（ 其中包含 NLP 相关文章），通过 Python 示例 进行了演示。我们通过 "语义文本相似性 "检查摘要是否抄袭，"语义文本相似性 "考虑的是用之前导入 Elasticsearch 的文本嵌入模型生成的 "摘要 "嵌入。此外，为了识别人工智能生成的内容--人工智能剽窃，还将 OpenAI 开发的NLP 模型导入 Elasticsearch。
下图说明了数据流：
在使用 推理处理器 的摄取管道 中，"抽象 "段落被映射为一个 768 维的向量，即 "abstract_vector.predicted_value"。
绘图：
"abstract_vector.predicted_value": { # Inference results field
"type": "dense_vector",
"dims": 768, # model embedding_size
"index": "true",
"similarity": "dot_product" # When indexing vectors for approximate kNN search, you need to specify the similarity function for comparing the vectors.矢量表示之间的相似性使用矢量相似性度量来衡量，该度量使用 "相似性 "参数来定义。
余弦是默认的相似度量，计算公式为"(1 + cosine(query, vector))/ 2'.除非您需要保留原始矢量，并且无法事先对它们进行归一化处理，否则执行余弦相似性的最有效方法就是将所有矢量归一化为单位长度。这有助于避免在搜索过程中执行额外的矢量长度计算，而应使用 "dot_product"。
在同一管道中，另一个包含文本分类模型的推理处理器会检测内容是可能由人类撰写的 "真实 "内容，还是可能由人工智能撰写的 "虚假 "内容，并将 "openai-detector.predicted_value "添加到每个文档中。
摄取管道：
client.ingest.put_pipeline(
id="plagiarism-checker-pipeline",
processors = [
{
"inference": { #for ml models - to infer against the data that is being ingested in the pipeline
"model_id": "roberta-base-openai-detector", #text classification model id
"target_field": "openai-detector", # Target field for the inference results
"field_map": { #Maps the document field names to the known field names of the model.
"abstract": "text_field" # Field matching our configured trained model input.
}
}
},
{
"inference": {
"model_id": "sentence-transformers__all-mpnet-base-v2", #text embedding model id
"target_field": "abstract_vector", # Target field for the inference results
"field_map": {
"abstract": "text_field" # Field matching our configured trained model input. Typically for NLP models, the field name is text_field.
}
}
}
]
)在查询时，同样的文本嵌入模型也被用于在 "query_vector_builder "对象中生成查询 "model_text "的向量表示。
k-nearest neighbor (kNN) 搜索会根据相似度量找到与查询向量最近的 k 个向量。
每份文档的 _ 分数都来自相似度，确保分数越大，排名越靠前。这意味着文件在语义上更加相似。因此，我们打印了三种可能性：如果得分> 0.9，我们认为是 "高度相似"；如果< 0.7，是 "低度相似"，否则是 "中度相似"。您可以根据自己的使用情况，灵活设置不同的阈值，以确定哪种程度的 _score 是否属于抄袭。
此外，还进行了文本分类，以检查文本查询中是否有人工智能生成的元素。
查询：
from elasticsearch import Elasticsearch
from elasticsearch.client import MlClient
#duplicated text - direct plagiarism test
model_text = 'Understanding and reasoning about cooking recipes is a fruitful research direction towards enabling machines to interpret procedural text. In this work, we introduce RecipeQA, a dataset for multimodal comprehension of cooking recipes. It comprises of approximately 20K instructional recipes with multiple modalities such as titles, descriptions and aligned set of images. With over 36K automatically generated question-answer pairs, we design a set of comprehension and reasoning tasks that require joint understanding of images and text, capturing the temporal flow of events and making sense of procedural knowledge. Our preliminary results indicate that RecipeQA will serve as a challenging test bed and an ideal benchmark for evaluating machine comprehension systems. The data and leaderboard are available at http://hucvl.github.io/recipeqa.'
response = client.search(index='plagiarism-checker', size=1,
knn={
"field": "abstract_vector.predicted_value",
"k": 9,
"num_candidates": 974,
"query_vector_builder": { #The 'all-mpnet-base-v2' model is also employed to generate the vector representation of the query in a 'query_vector_builder' object.
"text_embedding": {
"model_id": "sentence-transformers__all-mpnet-base-v2",
"model_text": model_text
}
}
}
)
for hit in response['hits']['hits']:
score = hit['_score']
title = hit['_source']['title']
abstract = hit['_source']['abstract']
openai = hit['_source']['openai-detector']['predicted_value']
url = hit['_source']['url']
if score > 0.9:
print(f"\nHigh similarity detected! This might be plagiarism.")
print(f"\nMost similar document: '{title}'\n\nAbstract: {abstract}\n\nurl: {url}\n\nScore:{score}\n\n")
if openai == 'Fake':
print("This document may have been created by AI.\n")
elif score < 0.7:
print(f"\nLow similarity detected. This might not be plagiarism.")
if openai == 'Fake':
print("This document may have been created by AI.\n")
else:
print(f"\nModerate similarity detected.")
print(f"\nMost similar document: '{title}'\n\nAbstract: {abstract}\n\nurl: {url}\n\nScore:{score}\n\n")
if openai == 'Fake':
print("This document may have been created by AI.\n")
ml_client = MlClient(client)
model_id = 'roberta-base-openai-detector' #open ai text classification model
document = [
{
"text_field": model_text
}
]
ml_response = ml_client.infer_trained_model(model_id=model_id, docs=document)
predicted_value = ml_response['inference_results'][0]['predicted_value']
if predicted_value == 'Fake':
print("\nNote: The text query you entered may have been generated by AI.\n")输出：
High similarity detected! This might be plagiarism.
Most similar document: 'RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes'
Abstract: Understanding and reasoning about cooking recipes is a fruitful research direction towards enabling machines to interpret procedural text. In this work, we introduce RecipeQA, a dataset for multimodal comprehension of cooking recipes. It comprises of approximately 20K instructional recipes with multiple modalities such as titles, descriptions and aligned set of images. With over 36K automatically generated question-answer pairs, we design a set of comprehension and reasoning tasks that require joint understanding of images and text, capturing the temporal flow of events and making sense of procedural knowledge. Our preliminary results indicate that RecipeQA will serve as a challenging test bed and an ideal benchmark for evaluating machine comprehension systems. The data and leaderboard are available at[ http://hucvl.github.io/recipeqa](http://hucvl.github.io/recipeqa).
url:[http://aclweb.org/anthology/D18-1166](http://aclweb.org/anthology/D18-1166)
Score:1.0在本例中，利用数据集中的一个 "摘要 "值作为文本查询 "model_text "后，剽窃行为被识别出来。相似度分数为 1.0，表明相似度很高，即直接抄袭。矢量化查询和文档没有被识别为人工智能生成的内容，这在意料之中。
查询：
#similar text - paraphrase plagiarism test
model_text = 'Comprehending and deducing information from culinary instructions represents a promising avenue for research aimed at empowering artificial intelligence to decipher step-by-step text. In this study, we present CuisineInquiry, a database for the multifaceted understanding of cooking guidelines. It encompasses a substantial number of informative recipes featuring various elements such as headings, explanations, and a matched assortment of visuals. Utilizing an extensive set of automatically crafted question-answer pairings, we formulate a series of tasks focusing on understanding and logic that necessitate a combined interpretation of visuals and written content. This involves capturing the sequential progression of events and extracting meaning from procedural expertise. Our initial findings suggest that CuisineInquiry is poised to function as a demanding experimental platform.'输出：
High similarity detected! This might be plagiarism.
Most similar document: 'RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes'
Abstract: Understanding and reasoning about cooking recipes is a fruitful research direction towards enabling machines to interpret procedural text. In this work, we introduce RecipeQA, a dataset for multimodal comprehension of cooking recipes. It comprises of approximately 20K instructional recipes with multiple modalities such as titles, descriptions and aligned set of images. With over 36K automatically generated question-answer pairs, we design a set of comprehension and reasoning tasks that require joint understanding of images and text, capturing the temporal flow of events and making sense of procedural knowledge. Our preliminary results indicate that RecipeQA will serve as a challenging test bed and an ideal benchmark for evaluating machine comprehension systems. The data and leaderboard are available at[ http://hucvl.github.io/recipeqa](http://hucvl.github.io/recipeqa).
url:[http://aclweb.org/anthology/D18-1166](http://aclweb.org/anthology/D18-1166)
Score:0.9302529
Note: The text query you entered may have been generated by AI.通过更新文本查询 "model_text"，使用人工智能生成的文本，在传达相同信息的同时尽量减少相似词语的重复，检测到的相似度仍然很高，但得分为 0.9302529，而不是 1.0，这就是转述剽窃。人们还预计，由人工智能生成的这一查询会被检测到。
最后，考虑到文本查询 "model_text "是关于 Elasticsearch 的文本，而不是这些文档的摘要，检测到的相似度为 0.68991005，表明根据所考虑的阈值，相似度较低。
查询：
#different text - not a plagiarism
model_text = 'Elasticsearch provides near real-time search and analytics for all types of data.'输出：
Low similarity detected. This might not be plagiarism.虽然在人工智能生成的文本查询中，以及在转述和直接复制内容的情况下，都能准确识别出抄袭行为，但在剽窃检测领域中，还需要认识到各个方面的问题。
在人工智能生成的内容检测方面，我们探索了一种模式，这种模式做出了宝贵的贡献。不过，必须认识到独立检测的固有局限性，因此需要结合其他方法来提高准确性。
选择文本嵌入模型带来的可变性是另一个考虑因素。使用不同数据集训练的不同模型会产生不同程度的相似性，这突出了所生成的文本嵌入的重要性。
最后，在这些例子中，我们使用了文件摘要。然而，剽窃检测通常涉及大量文件，因此必须解决文本长度的难题。文本通常会超过模型的标记限制，这就要求在建立嵌入之前将文本分割成块。处理这种情况的实用方法是使用 dense_vector 嵌套结构。
结论
在这篇博客中，我们讨论了检测抄袭（尤其是在转述和人工智能生成的内容中）所面临的挑战，以及如何利用语义文本相似性和文本分类来实现这一目的。
结合这些方法，我们提供了一个剽窃检测实例，成功识别了人工智能生成的内容、直接剽窃和转述剽窃。
我们的主要目标是建立一个过滤系统，简化检测工作，但人工评估对于验证工作仍然至关重要。
如果您有兴趣了解更多有关语义文本相似性和 NLP 的信息，我们建议您同时查看以下链接：
