Explore Elasticsearch data fields, such as sparse, dense & geo fields, and how to combine them to produce in-depth analytics on your data.

How to combine multiple sparse, dense and geo fields in custom ways

Sparse fields

Dense fields

Geo fields

Combining diverse fields

Creating "books" index

Indexing sample data

Finding Java books near a tech hub

Querying for special edition search technology books

Searching for latest IT books available in multiple languages

Wrap up

Dense fields are those expected to appear in all or most documents. They help provide solid search queries.

What are dense fields in Elasticsearch?

Sparse fields are fields that are not present in every document. Those fields are useful for filtering results based on attributes that only a subset of the dataset possesses.

What are sparse fields in Elasticsearch?

Elasticsearch sparse, dense & geo fields: How to combine them

Combining sparse, dense and geo fields in Elasticsearch

Learn how to search through documents with multiple kNN fields, and score the resultant documents based on multiple kNN vector fields.

Introduction to kNN search and semantic search

kNN mechanics

Scoring documents

How to search and score documents with multiple kNN fields

Text embedding and vector fields

Creating an Index with kNN Fields

Indexing sample docs

Setting up an inference processor

Indexing the documents

kNN search

Approximate search

Searching with multiple kNN fields

When to search with multi-kNN fields

Conclusion

kNN search: Elasticsearch scoring based on multiple kNN fields

Scoring documents based on the closest ones with multiple kNN fields

Learn strategies for selecting the optimal values for `k` and `num_candidates` parameters in kNN search, illustrated with practical examples.

How to choose the best `k` and `num_candidates` for kNN search?

kNN search query

kNN critical parameters - k and num_candidates

K: Unveiling the closest neighbors

Setting lower K

Setting higher K

The `num_candidates` attribute: Behind the curtain

Setting the lower num_candidates

Setting the higher num_candidates

Balancing precision & performance for kNN parameters

Experimentation to optimize kNN parameters

Practical example: Using kNN for movie recommendations

Manual framework

Creating an inference pipeline

Creating index mappings

Indexing movies

kNN query attributes

kNN results

Choosing optimal K value

Choosing the optimal `num_candidates` value

Finding the optimal balance

k refers to the number of global nearest neighbors to retrieve, and num_candidates to the number of candidate neighbors considered for each shard during the search.

What do the k and num_candidates parameters represent in kNN search?

If you prioritize extreme precision with a smaller set of highly relevant results, a lower k with a moderate num_candidates might be ideal. If exploration and discovery are your goals, a higher K with a larger num_candidates could be more suitable. By experimenting with different combinations and monitoring performance, you can achieve the perfect balance between precision, exploration, and speed.

How to choose the optimal value of the K and num_candidates parameters for KNN search?

Elasticsearch kNN search: How to choose the best k and num_candidates

How to choose the best k and num_candidates for kNN search

Author

Madhusudhan Konda

Articles

Combining sparse, dense and geo fields in Elasticsearch

Scoring documents based on the closest ones with multiple kNN fields

How to choose the best k and num_candidates for kNN search