Techniques avancées de RAG, partie 2 : Requêtes et tests
Discuter et mettre en œuvre des techniques susceptibles d'améliorer les performances du RAG. Partie 2 sur 2, axée sur l'interrogation et le test d'un pipeline RAG avancé.
Tout le code peut être trouvé dans le repo de Searchlabs, dans la branche advanced-rag-techniques.
Bienvenue dans la deuxième partie de notre article sur les techniques avancées de RAG ! Dans la première partie de cette série, nous avons mis en place, discuté et implémenté les composants de traitement des données du pipeline RAG avancé :
Le pipeline RAG utilisé par l'auteur.
Dans cette partie, nous allons procéder à l'interrogation et au test de notre mise en œuvre. Allons droit au but !
Table des matières
Recherche et récupération, génération de réponses
Posons notre première question, idéalement une information trouvée principalement dans le rapport annuel. Que diriez-vous de.. :
Who audits Elastic?"Appliquons maintenant quelques-unes de nos techniques pour améliorer la requête.
Enrichir les requêtes avec des synonymes
Tout d'abord, améliorons la diversité de la formulation de la requête et transformons-la en une forme qui peut être facilement traitée dans une requête Elasticsearch. Nous ferons appel à GPT-4o pour convertir la requête en une liste de clauses OR. Écrivons ce message :
ELASTIC_SEARCH_QUERY_GENERATOR_PROMPT = '''
You are an AI assistant specialized in generating Elasticsearch query strings. Your task is to create the most effective query string for the given user question. This query string will be used to search for relevant documents in an Elasticsearch index.
Guidelines:
1. Analyze the user's question carefully.
2. Generate ONLY a query string suitable for Elasticsearch's match query.
3. Focus on key terms and concepts from the question.
4. Include synonyms or related terms that might be in relevant documents.
5. Use simple Elasticsearch query string syntax if helpful (e.g., OR, AND).
6. Do not use advanced Elasticsearch features or syntax.
7. Do not include any explanations, comments, or additional text.
8. Provide only the query string, nothing else.
For the question "What is Clickthrough Data?", we would expect a response like:
clickthrough data OR click-through data OR click through rate OR CTR OR user clicks OR ad clicks OR search engine results OR web analytics
AND operator is not allowed. Use only OR.
User Question:
[The user's question will be inserted here]
Generate the Elasticsearch query string:
'''Appliqué à notre requête, GPT-4o génère des synonymes de la requête de base et du vocabulaire connexe.
'audits elastic OR
elasticsearch audits OR
elastic auditor OR
elasticsearch auditor OR
elastic audit firm OR
elastic audit company OR
elastic audit organization OR
elastic audit service'Dans la classe ESQueryMaker, j'ai défini une fonction pour diviser la requête :
def parse_or_query(self, query_text: str) -> List[str]:
# Split the query by 'OR' and strip whitespace from each term
# This converts a string like "term1 OR term2 OR term3" into a list ["term1", "term2", "term3"]
return [term.strip() for term in query_text.split(' OR ')]Son rôle est de prendre cette chaîne de clauses OR et de les diviser en une liste de termes, ce qui nous permet d'effectuer une correspondance multiple sur les champs clés du document :
["original_text", 'keyphrases', 'potential_questions', 'entities']Finalement, nous avons abouti à cette requête :
'query': {
'bool': {
'must': [
{
'multi_match': {
'query': 'audits Elastic Elastic auditing Elastic audit process Elastic compliance Elastic security audit Elasticsearch auditing Elasticsearch compliance Elasticsearch security audit',
'fields': [
'original_text',
'keyphrases',
'potential_questions',
'entities'
],
'type': 'best_fields',
'operator': 'or'
}
}
]Cela permet de couvrir beaucoup plus de bases que la requête initiale, réduisant ainsi le risque de manquer un résultat de recherche en raison de l'oubli d'un synonyme. Mais nous pouvons faire plus.
HyDE (Hypothetical Document Embedding)
Faisons à nouveau appel à GPT-4o, cette fois pour mettre en œuvre HyDE.
Le principe de base de HyDE est de générer un document hypothétique - le type de document qui contiendrait probablement la réponse à la requête initiale. Le caractère factuel ou l'exactitude du document n'est pas en cause. En gardant cela à l'esprit, rédigeons l'invite suivante :
HYDE_DOCUMENT_GENERATOR_PROMPT = '''
You are an AI assistant specialized in generating hypothetical documents based on user queries. Your task is to create a detailed, factual document that would likely contain the answer to the user's question. This hypothetical document will be used to enhance the retrieval process in a Retrieval-Augmented Generation (RAG) system.
Guidelines:
1. Carefully analyze the user's query to understand the topic and the type of information being sought.
2. Generate a hypothetical document that:
a. Is directly relevant to the query
b. Contains factual information that would answer the query
c. Includes additional context and related information
d. Uses a formal, informative tone similar to an encyclopedia or textbook entry
3. Structure the document with clear paragraphs, covering different aspects of the topic.
4. Include specific details, examples, or data points that would be relevant to the query.
5. Aim for a document length of 200-300 words.
6. Do not use citations or references, as this is a hypothetical document.
7. Avoid using phrases like "In this document" or "This text discusses" - write as if it's a real, standalone document.
8. Do not mention or refer to the original query in the generated document.
9. Ensure the content is factual and objective, avoiding opinions or speculative information.
10. Output only the generated document, without any additional explanations or meta-text.
User Question:
[The user's question will be inserted here]
Generate a hypothetical document that would likely contain the answer to this query:
'''La recherche vectorielle s'appuyant généralement sur la similarité vectorielle cosinusoïdale, HyDE part du principe que l'on peut obtenir de meilleurs résultats en faisant correspondre des documents à des documents plutôt que des requêtes à des documents.
Ce qui nous importe, c'est la structure, le flux et la terminologie. Ce n'est pas tant le cas pour les faits. Le GPT-4o produit un document HyDE comme celui-ci :
'Elastic N.V., the parent company of Elastic, the organization known for developing Elasticsearch, is subject to audits to ensure financial accuracy,
regulatory compliance, and the integrity of its financial statements. The auditing of Elastic N.V. is typically conducted by an external,
independent auditing firm. This is common practice for publicly traded companies to provide stakeholders with assurance regarding the company\'s
financial position and operations.\n\nThe primary external auditor for Elastic is the audit firm Ernst & Young LLP (EY). Ernst & Young is one of the
four largest professional services networks in the world, commonly referred to as the "Big Four" audit firms. These firms handle a substantial number
of audits for major corporations around the globe, ensuring adherence to generally accepted accounting principles (GAAP) and international financial
reporting standards (IFRS).\n\nThe audit process conducted by EY involves several steps. Initially, the auditors perform a risk assessment to identify
areas where misstatements due to error or fraud could occur. They then design audit procedures to test the accuracy and completeness of financial statements,
which include examining financial transactions, assessing internal controls, and reviewing compliance with relevant laws and regulations. Upon completion of
the audit, Ernst & Young issues an audit report, which includes the auditor’s opinion on whether the financial statements are free from material misstatement
and are presented fairly in accordance with the applicable financial reporting framework.\n\nIn addition to external audits by firms like Ernst & Young,
Elastic may also be subject to internal audits. Internal audits are performed by the company’s own internal auditors to evaluate the effectiveness of internal
controls, risk management, and governance processes.\n\nOverall, the auditing process plays a crucial role in maintaining the transparency and reliability of
Elastic\'s financial information, providing confidence to investors, regulators, and other stakeholders.'Il semble assez crédible, comme le candidat idéal pour les types de documents que nous aimerions indexer. Nous allons l'intégrer et l'utiliser pour la recherche hybride.
Recherche hybride
C'est le cœur de notre logique de recherche. Notre composante de recherche lexicale sera les chaînes de clauses OR générées. Notre composante vectorielle dense sera le document HyDE intégré (également appelé vecteur de recherche). Nous utilisons KNN pour identifier efficacement les documents candidats les plus proches de notre vecteur de recherche. Nous appelons notre composant de recherche lexicale Scoring with TF-IDF and BM25 par défaut. Enfin, les scores des vecteurs lexicaux et denses seront combinés en utilisant le ratio 30/70 recommandé par Wang et al.
def hybrid_vector_search(self, index_name: str, query_text: str, query_vector: List[float],
text_fields: List[str], vector_field: str,
num_candidates: int = 100, num_results: int = 10) -> Dict:
"""
Perform a hybrid search combining text-based and vector-based similarity.
Args:
index_name (str): The name of the Elasticsearch index to search.
query_text (str): The text query string, which may contain 'OR' separated terms.
query_vector (List[float]): The query vector for semantic similarity search.
text_fields (List[str]): List of text fields to search in the index.
vector_field (str): The name of the field containing document vectors.
num_candidates (int): Number of candidates to consider in the initial KNN search.
num_results (int): Number of final results to return.
Returns:
Dict: A tuple containing the Elasticsearch response and the search body used.
"""
try:
# Parse the query_text into a list of individual search terms
# This splits terms separated by 'OR' and removes any leading/trailing whitespace
query_terms = self.parse_or_query(query_text)
# Construct the search body for Elasticsearch
search_body = {
# KNN search component for vector similarity
"knn": {
"field": vector_field, # The field containing document vectors
"query_vector": query_vector, # The query vector to compare against
"k": num_candidates, # Number of nearest neighbors to retrieve
"num_candidates": num_candidates # Number of candidates to consider in the KNN search
},
"query": {
"bool": {
# The 'must' clause ensures that matching documents must satisfy this condition
# Documents that don't match this clause are excluded from the results
"must": [
{
# Multi-match query to search across multiple text fields
"multi_match": {
"query": " ".join(query_terms), # Join all query terms into a single space-separated string
"fields": text_fields, # List of fields to search in
"type": "best_fields", # Use the best matching field for scoring
"operator": "or" # Match any of the terms (equivalent to the original OR query)
}
}
],
# The 'should' clause boosts relevance but doesn't exclude documents
# It's used here to combine vector similarity with text relevance
"should": [
{
# Custom scoring using a script to combine vector and text scores
"script_score": {
"query": {"match_all": {}}, # Apply this scoring to all documents that matched the 'must' clause
"script": {
# Script to combine vector similarity and text relevance
"source": """
# Calculate vector similarity (cosine similarity + 1)
# Adding 1 ensures the score is always positive
double vector_score = cosineSimilarity(params.query_vector, params.vector_field) + 1.0;
# Get the text-based relevance score from the multi_match query
double text_score = _score;
# Combine scores: 70% vector similarity, 30% text relevance
# This weighting can be adjusted based on the importance of semantic vs keyword matching
return 0.7 * vector_score + 0.3 * text_score;
""",
# Parameters passed to the script
"params": {
"query_vector": query_vector, # Query vector for similarity calculation
"vector_field": vector_field # Field containing document vectors
}
}
}
}
]
}
}
}
# Execute the search request against the Elasticsearch index
response = self.conn.search(index=index_name, body=search_body, size=num_results)
# Log the successful execution of the search for monitoring and debugging
logger.info(f"Hybrid search executed on index: {index_name} with text query: {query_text}")
# Return both the response and the search body (useful for debugging and result analysis)
return response, search_body
except Exception as e:
# Log any errors that occur during the search process
logger.error(f"Error executing hybrid search on index: {index_name}. Error: {e}")
# Re-raise the exception for further handling in the calling code
raise eEnfin, nous pouvons reconstituer une fonction RAG. Notre RAG, de la question à la réponse, suivra ce flux :
Convertir la requête en clauses OR.
Générer un document HyDE et l'intégrer.
Transmettre ces deux informations à la recherche hybride.
Récupérer les n premiers résultats, les inverser de façon à ce que le score le plus pertinent soit le "plus récent" dans la mémoire contextuelle du LLM (Reverse Packing) Reverse Packing Exemple : Requête : "Techniques d'optimisation des requêtes Elasticsearch" Documents récupérés (ordonnés par pertinence) : Ordre inversé pour le contexte LLM : En inversant l'ordre, l'information la plus pertinente (1) apparaît en dernier dans le contexte, recevant potentiellement plus d'attention de la part du LLM pendant la génération de la réponse.
"Utilisez les requêtes bool pour combiner efficacement plusieurs critères de recherche."
"Mettre en œuvre des stratégies de mise en cache pour améliorer les temps de réponse des requêtes."
"Optimiser les mappages d'index pour accélérer les performances de recherche."
"Optimiser les mappages d'index pour accélérer les performances de recherche."
"Mettre en œuvre des stratégies de mise en cache pour améliorer les temps de réponse des requêtes."
"Utilisez les requêtes bool pour combiner efficacement plusieurs critères de recherche."
Transmettre le contexte au LLM pour qu'il le génère.
def get_context(index_name,
match_query,
text_query,
fields,
num_candidates=100,
num_results=20,
text_fields=["original_text", 'keyphrases', 'potential_questions', 'entities'],
embedding_field="primary_embedding"):
embedding=embedder.get_embeddings_from_text(text_query)
results, search_body = es_query_maker.hybrid_vector_search(
index_name=index_name,
query_text=match_query,
query_vector=embedding[0][0],
text_fields=text_fields,
vector_field=embedding_field,
num_candidates=num_candidates,
num_results=num_results
)
# Concatenates the text in each 'field' key of the search result objects into a single block of text.
context_docs=['\n\n'.join([field+":\n\n"+j['_source'][field] for field in fields]) for j in results['hits']['hits']]
# Reverse Packing to ensure that the highest ranking document is seen first by the LLM.
context_docs.reverse()
return context_docs, search_body
def retrieval_augmented_generation(query_text):
match_query= gpt4o.generate_query(query_text)
fields=['original_text']
hyde_document=gpt4o.generate_HyDE(query_text)
context, search_body=get_context(index_name, match_query, hyde_document, fields)
answer= gpt4o.basic_qa(query=query_text, context=context)
return answer, match_query, hyde_document, context, search_bodyExécutons notre requête et obtenons notre réponse :
According to the context, Elastic N.V. is audited by an independent registered public accounting firm, PricewaterhouseCoopers (PwC).
This information is found in the section titled "report of independent registered public accounting firm," which states:
"We have audited the accompanying consolidated balance sheets of Elastic N.V. [...] / s / pricewaterhouseco."C'est une bonne chose. C'est exact.
Expériences
Il faut maintenant répondre à une question importante. Qu'avons-nous obtenu en investissant tant d'efforts et de complexité supplémentaire dans ces mises en œuvre ?
Faisons une petite comparaison. Le pipeline RAG que nous avons mis en œuvre par rapport à la recherche hybride de base, sans aucune des améliorations que nous avons apportées. Nous allons effectuer une petite série de tests et voir si nous constatons des différences substantielles. Nous appellerons le RAG que nous venons de mettre en œuvre AdvancedRAG et le pipeline de base SimpleRAG.
RAG Pipeline simple et sans fioritures
Synthèse des résultats
Ce tableau résume les résultats de cinq tests effectués sur les deux pipelines RAG. J'ai évalué la supériorité relative de chaque méthode sur la base du détail et de la qualité des réponses, mais il s'agit d'un jugement totalement subjectif. Les réponses réelles sont reproduites sous ce tableau pour votre considération. Ceci étant dit, jetons un coup d'œil sur leurs résultats !
SimpleRAG n'a pas pu répondre aux questions 1 & 5. AdvancedRAG a également répondu de manière beaucoup plus détaillée aux questions 2, 3 et 4. Sur la base de ces détails, j'ai jugé la qualité des réponses d'AdvancedRAG meilleure.
Test | Question | AdvancedRAG Performance | Performance de SimpleRAG | AdvancedRAG Latence | Latence de SimpleRAG | Gagnant |
|---|---|---|---|---|---|---|
1 | Qui audite Elastic ? | A correctement identifié PwC comme étant l'auditeur. | L'auditeur n'a pas été identifié. | 11.6s | 4.4s | AdvancedRAG |
2 | Quel a été le revenu total en 2023 ? | A fourni le chiffre d'affaires correct. Inclusion d'un contexte supplémentaire avec les recettes des années précédentes. | A fourni le chiffre d'affaires correct. | 13.3s | 2.8s | AdvancedRAG |
3 | De quel produit la croissance dépend-elle principalement ? Combien ? | A correctement identifié l'Elastic Cloud comme étant le facteur clé. Le contexte général des recettes a été inclus & de manière plus détaillée. | A correctement identifié l'Elastic Cloud comme étant le facteur clé. | 14.1s | 12.8s | AdvancedRAG |
4 | Décrire le régime d'avantages sociaux des employés | Description complète des régimes de retraite, des programmes de santé et des autres avantages. Incluait des montants de cotisation spécifiques pour différentes années. | A donné un bon aperçu des avantages, y compris la rémunération, les plans de retraite, l'environnement de travail et le programme Elastic Cares. | 26.6s | 11.6s | AdvancedRAG |
5 | Quelles sont les entreprises acquises par Elastic ? | A corrigé la liste des acquisitions récentes mentionnées dans le rapport (CmdWatch, Build Security, Optimyze). Il a fourni quelques dates d'acquisition et des prix d'achat. | Échec de la recherche d'informations pertinentes dans le contexte fourni. | 11.9s | 2.7s | AdvancedRAG |
Test 1 : Qui vérifie Elastic ?
AdvancedRAG
According to the context, Elastic N.V. is audited by an independent
registered public accounting firm, PricewaterhouseCoopers (PwC).
This information is found in the section titled "report of
independent registered public accounting firm," which states:
"We have audited the accompanying consolidated balance sheets
of Elastic N.V. [...] / s / pricewaterhouseco."SimpleRAG
According to the provided context, no specific auditing firm
or company is mentioned as the auditor for Elastic.
The context provided talks about various features, strengths,
deployment options, and other details related to Elastic and
its operations but does not specify which auditing firm is
responsible for auditing Elastic's financial statements or
internal controls.Résumé: SimpleRAG n'a pas identifié PWC comme étant l'auditeur.
D'accord, c'est assez surprenant. Cela ressemble à un échec de recherche de la part de SimpleRAG. Aucun document relatif à l'audit n'a été retrouvé. Réduisons un peu la difficulté avec le test suivant.
Test 2 : recettes totales en 2023
AdvancedRAG
### Total Revenue for 2023
According to the provided context, the total revenue for the
year ended April 30, 2023, was **$1,068,989,000**. This
information can be found several times in the provided text.
Specifically, one part states:
"The following table sets forth our results of operations
for the periods presented in dollars and as a percentage of
our total revenue. ... year ended April 30, 2023, ...
total revenue 1,068,989".
Additionally, it is mentioned here as well:
"For the years ended April 30, 2023, 2022, and 2021,
representing year-over-year growth of 24% for the year
ended April 30, 2023... total revenue $1,068,989".
Therefore, the total revenue for 2023 is $1.069 billion.SimpleRAG
According to the context provided, the total revenue for the fiscal year ended April 30, 2023, is:
$1,068,989,000 (or approximately $1.1 billion).Résumé: les deux RAG ont obtenu la bonne réponse : 1 068 989 000 dollars de recettes totales en 2023.
Ils étaient tous les deux ici. Il semble qu'AdvancedRAG ait acquis un plus grand nombre de documents ? La réponse est certainement plus détaillée et intègre des informations des années précédentes. On peut s'y attendre compte tenu des améliorations que nous avons apportées, mais il est encore trop tôt pour se prononcer.
Augmentons la difficulté.
Test 3 : De quel produit la croissance dépend-elle principalement ? Combien ?
AdvancedRAG
### Product Growth Primarily Depends On:
According to the context, "our revenue is derived primarily from
subscriptions of search, observability and security built into
the elastic stack." The document mentions various strategies and
factors contributing to the company’s growth which include investment
in "elastic cloud," "elastic stack," and the conversion of free
users to paid subscribers.
### Quantitative Data:
- **Revenue**: The document states, "Our total revenue was $1.1 billion,
$862.4 million, and $608.5 million for the years ended April 30, 2023,
2022, and 2021, respectively."
- **Subscription Revenue**: "Subscriptions accounted for 92%, 93%,
and 93% of our total revenue for the years ended April 30, 2023,
2022, and 2021, respectively."
- **Elastic Cloud Contribution**: "For the years ended April 30, 2023,
2022, and 2021, Elastic Cloud contributed 40%, 35%, and 27% of our
total revenue, respectively."
- **Customer Growth**: The number of customers increased from
"over 15,000 as of April 30, 2021," to "approximately 20,200 as of
April 30, 2023."
### Summary:
Product growth for the company primarily depends on the "Elastic Stack"
(which includes search, observability, and security solutions) and
particularly the "Elastic Cloud." The Elastic Cloud's growing contribution
to revenue highlights its significance, which grew from 27% in 2021 to 40% in 2023.SimpleRAG
According to the provided context, growth primarily depends on **Elastic Cloud**,
which is described as the company's "family of cloud-based offerings."
The context also emphasizes the significance of Elastic Cloud in achieving
the company's long-term growth potential and its role in increasing subscription revenue.
From the context:
- "Increasing adoption of Elastic Cloud... is an important growth opportunity
for our business. Organizations are increasingly looking for hosted
deployment alternatives with reduced administrative burdens."
- "We believe that offering Elastic Cloud is important for achieving
our long-term growth potential, and we expect Elastic Cloud’s contribution
to our subscription revenue to continue to increase."
In terms of the contribution to total revenue, here are the specific figures provided:
- Elastic Cloud contributed 40% of the total revenue for the year ended April 30, 2023.
- Elastic Cloud contributed 35% of the total revenue for the year ended April 30, 2022.
- Elastic Cloud contributed 27% of the total revenue for the year ended April 30, 2021.
This progression shows a clear upward trend in the importance and
contribution of Elastic Cloud to the company's overall revenue.Résumé: les deux groupes d'experts ont correctement identifié Elastic Cloud comme le principal moteur de croissance. Cependant, AdvancedRAG fournit plus de détails, en tenant compte des revenus d'abonnement et de la croissance de la clientèle, et mentionne explicitement d'autres offres d'Elastic.
Test 4 : Décrire le régime d'avantages sociaux des salariés
AdvancedRAG
### Employee Benefit Plan Overview
The provided context outlines various elements of the employee benefit plan at
Elastic, often referred to as "Elasticians." Here are the key components
described in the texts:
1. **401(k) Plan**:
- **Coverage**: The 401(k) plan is available to substantially all U.S.
employees who meet minimum age and service requirements.
- **Contributions**: Elastic makes contributions to the 401(k) plan up to
6% of the participating employee’s W-2 earnings and wages.
- **Expenses**: For the fiscal years ended April 30, Elastic recorded
expenses of $17.9 million (2023), $15.2 million (2022), and $11.4 million (2021)
related to the 401(k) plan.
- **Defined-Contribution Plans in Other Countries**: Elastic has
defined-contribution plans in various other countries and recorded respective
expenses of $9.4 million (2023), $7.2 million (2022), and $5.1 million (2021).
2. **Stock-Based Compensation**:
- **Types of Awards**: Stock options, restricted stock units (RSUs),
and shares under the Employee Stock Purchase Plan (ESPP).
- **Fair Value Measurement**: Fair value of these stock awards is
measured using models like Black-Scholes.
- **Employee Stock Purchase Plan (2022 ESPP)**:
- Started in 2022, it allows employees to acquire ordinary
shares at a discount (85% of the market value at the beginning
or end of the offering period).
- Offering periods are approximately six months long.
3. **Total Rewards Compensation**:
- **Components**: Includes cash compensation as well as equity awards,
reflecting a comprehensive interest in the well-being of the employees.
- **Programs**: Market competitive medical and dental programs,
mental health, and holistic well-being programs.
- **Paid Time Off**: Features 16 weeks of paid leave for new
parents and life-planning benefits.
4. **Retirement and Income Protection Plans**:
- **401(k) Match**: Elastic provides a dollar-for-dollar
match up to 6% of eligible earnings up to a plan-limit
maximum for U.S.-based employees.
- **Similar Plans Outside the U.S.**: Competitive retirement
plans available for employees based outside the United States.
5. **Flexible Work Environment**:
- **Distributed Workforce**: Elastic operates with a global
distributed workforce model, allowing employees to work remotely
and providing a broad candidate pool.
6. **Learning and Development**:
- **Programs**: Access to on-demand video-based learning,
leadership development programs, and other specific training
to develop managers and leaders.
7. **Employee Engagement and Surveys**:
- **Mechanisms**: Annual employee engagement survey and
mid-year pulse survey.
- **Objectives**: To monitor morale, develop action plans,
and maintain high engagement scores.
8. **Onboarding Program**:
- **X-School**: An extensive new-hire orientation program to
help new employees meet peers and learn about the company's
products and solutions.
In summary, Elastic's employee benefit plan encompasses a broad
spectrum of financial, health, well-being, and professional development
benefits designed to support and enhance the lives of its employees.
The plan is both competitive and comprehensive, aiming to attract,
retain, and engage its global workforce.SimpleRAG
The employee benefit plan at the Company includes several key
components aimed at enhancing the overall well-being and
professional development of employees, referred to as "Elasticians."
1. **Compensation and Benefits:**
- The Company provides market-competitive compensation
packages which typically include cash compensation and equity awards.
- Comprehensive medical and dental programs are offered,
emphasizing mental health and holistic well-being.
- Paid time off programs feature:
- 16 weeks of paid leave for new parents.
- Life-planning benefits.
- Travel reimbursements for certain healthcare services.
2. **Retirement and Income Protection Plans:**
- In the United States, employees are eligible for a 401k
plan with a dollar-for-dollar match by the Company up to 6%
of eligible earnings, subject to a plan limit.
- Similar competitive retirement and income protection plans
are available for employees outside the United States.
3. **Work Environment:**
- The Company offers a flexible work environment, allowing most
employees to work from anywhere as often as they would like.
- Physical offices around the world are available for employees
who prefer to work on-site or need a space for face-to-face interactions.
4. **Community Involvement:**
- The Elastic Cares Program supports charitable organizations
that matter to employees on both local and global levels.
- The program includes:
- Donation matching.
- Providing the Company's technology for free to certain
nonprofit organizations.
- Volunteer time off initiative.Résumé: AdvancedRAG va beaucoup plus en profondeur et en détail, en mentionnant le plan 401K pour les employés basés aux États-Unis, ainsi qu'en définissant les plans de contribution en dehors des États-Unis. Il mentionne également les régimes de santé et de bien-être, mais ne mentionne pas le programme Elastic Cares, que SimpleRAG mentionne.
Test 5 : Quelles sont les entreprises acquises par Elastic ?
AdvancedRAG
Elastic acquired several companies over the years to enhance its
technology and market presence. According to the provided context,
the specific acquisitions include:
1. **CmdWatch Security Inc.**: Acquired on September 17, 2021.
The total purchase consideration was $77.8 million.
2. **Build Security Ltd.**: Acquired on September 2, 2021. The
combined purchase price for Build Security Ltd. and another acquisition was $57.2 million.
3. **Optimyze Cloud Inc.**: Acquired on November 1, 2021. Part
of the combined purchase price mentioned above with Build Security Ltd.
4. **Endgame, Inc.**: Although the context mentions an amended
and restated stock incentive plan, it indicates that Endgame, Inc. was
associated with Elastic, suggesting it could have been acquired at some point.
These acquisitions are part of Elastic's strategy to
enhance the technology underlying its security and
observability offerings.SimpleRAG
The provided context does not specify any companies that Elastic has acquired.
Therefore, based on the context, there is no information available about the companies acquired by Elastic.Résumé: SimpleRAG ne récupère aucune information pertinente sur les acquisitions, ce qui entraîne un échec de la réponse. AdvancedRAG répertorie correctement CmdWatch, Build Security et Optimyze, qui sont les principales acquisitions mentionnées dans le rapport.
Conclusion
D'après nos tests, nos techniques avancées semblent augmenter l'étendue et la profondeur des informations présentées, ce qui pourrait améliorer la qualité des réponses RAG.
En outre, il est possible que la fiabilité soit améliorée, car les questions formulées de manière ambiguë, telles que Which companies did Elastic acquire? et Who audits Elastic, ont été correctement répondues par AdvancedRAG, mais pas par SimpleRAG.
Toutefois, il convient de garder à l'esprit que dans 3 cas sur 5, le pipeline RAG de base, intégrant la recherche hybride mais aucune autre technique, a réussi à produire des réponses qui capturaient la plupart des informations clés.
Il convient de noter qu'en raison de l'incorporation des LLM dans les phases de préparation des données et d'interrogation, la latence d'AdvancedRAG est généralement de 2 à 5 fois supérieure à celle de SimpleRAG. Il s'agit d'un coût important qui pourrait faire en sorte qu'AdvancedRAG ne convienne qu'aux situations où la qualité de la réponse est prioritaire par rapport à la latence.
Les coûts de latence importants peuvent être réduits en utilisant un LLM plus petit et moins cher comme Claude Haiku ou GPT-4o-mini à l'étape de préparation des données. Conservez les modèles avancés pour la génération de réponses.
Cela correspond aux conclusions de Wang et al. Comme le montrent leurs résultats, les améliorations apportées sont relativement progressives. En bref, un simple RAG de base vous permet d'obtenir un produit final décent, tout en étant moins cher et plus rapide. Pour moi, c'est une conclusion intéressante. Pour les cas d'utilisation où la vitesse et l'efficacité sont essentielles, SimpleRAG est un choix judicieux. Pour les cas d'utilisation où chaque goutte de performance doit être extraite, les techniques incorporées dans AdvancedRAG peuvent offrir une solution.
Les résultats de l'étude de Wang et al révèlent que l'utilisation de techniques avancées entraîne des améliorations constantes mais progressives.
Annexe
Prompts
Question du RAG - invite à répondre
Invitation à faire en sorte que le LLM génère des réponses basées sur la requête et le contexte.
BASIC_RAG_PROMPT = '''
You are an AI assistant tasked with answering questions based primarily on the provided context, while also drawing on your own knowledge when appropriate. Your role is to accurately and comprehensively respond to queries, prioritizing the information given in the context but supplementing it with your own understanding when beneficial. Follow these guidelines:
1. Carefully read and analyze the entire context provided.
2. Primarily focus on the information present in the context to formulate your answer.
3. If the context doesn't contain sufficient information to fully answer the query, state this clearly and then supplement with your own knowledge if possible.
4. Use your own knowledge to provide additional context, explanations, or examples that enhance the answer.
5. Clearly distinguish between information from the provided context and your own knowledge. Use phrases like "According to the context..." or "The provided information states..." for context-based information, and "Based on my knowledge..." or "Drawing from my understanding..." for your own knowledge.
6. Provide comprehensive answers that address the query specifically, balancing conciseness with thoroughness.
7. When using information from the context, cite or quote relevant parts using quotation marks.
8. Maintain objectivity and clearly identify any opinions or interpretations as such.
9. If the context contains conflicting information, acknowledge this and use your knowledge to provide clarity if possible.
10. Make reasonable inferences based on the context and your knowledge, but clearly identify these as inferences.
11. If asked about the source of information, distinguish between the provided context and your own knowledge base.
12. If the query is ambiguous, ask for clarification before attempting to answer.
13. Use your judgment to determine when additional information from your knowledge base would be helpful or necessary to provide a complete and accurate answer.
Remember, your goal is to provide accurate, context-based responses, supplemented by your own knowledge when it adds value to the answer. Always prioritize the provided context, but don't hesitate to enhance it with your broader understanding when appropriate. Clearly differentiate between the two sources of information in your response.
Context:
[The concatenated documents will be inserted here]
Query:
[The user's question will be inserted here]
Please provide your answer based on the above guidelines, the given context, and your own knowledge where appropriate, clearly distinguishing between the two:
'''Générateur de requêtes élastiques
Invite à enrichir les requêtes avec des synonymes et à les convertir au format OR.
ELASTIC_SEARCH_QUERY_GENERATOR_PROMPT = '''
You are an AI assistant specialized in generating Elasticsearch query strings. Your task is to create the most effective query string for the given user question. This query string will be used to search for relevant documents in an Elasticsearch index.
Guidelines:
1. Analyze the user's question carefully.
2. Generate ONLY a query string suitable for Elasticsearch's match query.
3. Focus on key terms and concepts from the question.
4. Include synonyms or related terms that might be in relevant documents.
5. Use simple Elasticsearch query string syntax if helpful (e.g., OR, AND).
6. Do not use advanced Elasticsearch features or syntax.
7. Do not include any explanations, comments, or additional text.
8. Provide only the query string, nothing else.
For the question "What is Clickthrough Data?", we would expect a response like:
clickthrough data OR click-through data OR click through rate OR CTR OR user clicks OR ad clicks OR search engine results OR web analytics
AND operator is not allowed. Use only OR.
User Question:
[The user's question will be inserted here]
Generate the Elasticsearch query string:
'''Questions potentielles : invite à la création d'un générateur
Invite à générer des questions potentielles, à enrichir les métadonnées des documents.
RAG_QUESTION_GENERATOR_PROMPT = '''
You are an AI assistant specialized in generating questions for Retrieval-Augmented Generation (RAG) systems. Your task is to analyze a given document and create 10 diverse questions that would effectively test a RAG system's ability to retrieve and synthesize information from this document.
Guidelines:
1. Thoroughly analyze the entire document.
2. Generate exactly 10 questions that cover various aspects and levels of complexity within the document's content.
3. Create questions that specifically target:
a. Key facts and information
b. Main concepts and ideas
c. Relationships between different parts of the content
d. Potential applications or implications of the information
e. Comparisons or contrasts within the document
4. Ensure questions require answers of varying lengths and complexity, from simple retrieval to more complex synthesis.
5. Include questions that might require combining information from different parts of the document.
6. Frame questions to test both literal comprehension and inferential understanding.
7. Avoid yes/no questions; focus on open-ended questions that promote comprehensive answers.
8. Consider including questions that might require additional context or knowledge to fully answer, to test the RAG system's ability to combine retrieved information with broader knowledge.
9. Number the questions from 1 to 10.
10. Output only the ten questions, without any additional text, explanations, or answers.
Document:
[The document content will be inserted here]
Generate 10 questions optimized for testing a RAG system based on this document:
'''Invite du générateur HyDE
Invite à générer des documents hypothétiques à l'aide de HyDE
HYDE_DOCUMENT_GENERATOR_PROMPT = '''
You are an AI assistant specialized in generating hypothetical documents based on user queries. Your task is to create a detailed, factual document that would likely contain the answer to the user's question. This hypothetical document will be used to enhance the retrieval process in a Retrieval-Augmented Generation (RAG) system.
Guidelines:
1. Carefully analyze the user's query to understand the topic and the type of information being sought.
2. Generate a hypothetical document that:
a. Is directly relevant to the query
b. Contains factual information that would answer the query
c. Includes additional context and related information
d. Uses a formal, informative tone similar to an encyclopedia or textbook entry
3. Structure the document with clear paragraphs, covering different aspects of the topic.
4. Include specific details, examples, or data points that would be relevant to the query.
5. Aim for a document length of 200-300 words.
6. Do not use citations or references, as this is a hypothetical document.
7. Avoid using phrases like "In this document" or "This text discusses" - write as if it's a real, standalone document.
8. Do not mention or refer to the original query in the generated document.
9. Ensure the content is factual and objective, avoiding opinions or speculative information.
10. Output only the generated document, without any additional explanations or meta-text.
User Question:
[The user's question will be inserted here]
Generate a hypothetical document that would likely contain the answer to this query:
'''Exemple de requête de recherche hybride
{'knn': {'field': 'primary_embedding',
'query_vector': [0.4265527129173279,
-0.1712949573993683,
-0.042020395398139954,
...],
'k': 100,
'num_candidates': 100},
'query': {'bool': {'must': [{'multi_match': {'query': 'audits Elastic Elastic auditing Elastic audit process Elastic compliance Elastic security audit Elasticsearch auditing Elasticsearch compliance Elasticsearch security audit',
'fields': ['original_text',
'keyphrases',
'potential_questions',
'entities'],
'type': 'best_fields',
'operator': 'or'}}],
'should': [{'script_score': {'query': {'match_all': {}},
'script': {'source': '\n double vector_score = cosineSimilarity(params.query_vector, params.vector_field) + 1.0;\n double text_score = _score;\n return 0.7 * vector_score + 0.3 * text_score;\n ',
'params': {'query_vector': [0.4265527129173279,
-0.1712949573993683,
-0.042020395398139954,
...],
'vector_field': 'primary_embedding'}}}}]}},
'size': 10}
