references/hybrid-search.md
# Hybrid Search
This guide covers hybrid search patterns in MongoDB Atlas: combining vector and lexical search using `$rankFusion` and `$scoreFusion`, and using lexical prefilters with the `vectorSearch` operator inside `$search`.
**Scope**: This guide covers hybrid pipelines. For pure vector search indexes and `$vectorSearch` query construction, see vector-search.md. For lexical index definitions and query patterns, see lexical-search-indexing.md and lexical-search-querying.md.
## Table of Contents
- [Overview](#overview)
- [Choosing the Right Approach](#choosing-the-right-approach)
- [Indexing for Hybrid Search](#indexing-for-hybrid-search)
- [$rankFusion](#rankfusion)
- [$scoreFusion](#scorefusion)
- [Lexical Prefilters (vectorSearch Operator)](#lexical-prefilters-vectorsearch-operator)
- [Best Practices and Limitations](#best-practices-and-limitations)
---
## Overview
Hybrid search combines multiple search methods on the same collection and merges the results into a single ranked or scored list.
**Three patterns covered in this guide:**
| Pattern | Stage / Operator | Use When |
|---|---|---|
| Rank-based fusion | `$rankFusion` | Document position matters; use RRF algorithm |
| Score-based fusion | `$scoreFusion` | Score magnitude matters; need custom math or normalization |
| Lexical prefilter | `$search` + `vectorSearch` operator | Need fuzzy/phrase/wildcard/compound pre-filtering before vector search |
**$rankFusion vs $scoreFusion:**
- `$rankFusion` ranks by position in each input pipeline using the Reciprocal Rank Fusion (RRF) algorithm. A document ranked #1 in multiple pipelines scores much higher than one ranked #1 in only one. Weights influence how much each pipeline's rank contributes.
- `$scoreFusion` ranks by the actual score values from each pipeline. Supports normalization (sigmoid, minMaxScaler) and custom combination expressions. Use when score magnitude, not just ordering, matters.
---
## Choosing the Right Approach
| Scenario | Recommended Approach |
|---|---|
| Combine lexical + vector, rank by position | `$rankFusion` |
| Combine lexical + vector, control score math or normalization | `$scoreFusion` |
| Multiple query vectors or embedding models on same collection | `$rankFusion` with multiple `$vectorSearch` pipelines |
| Pre-filter vector search with fuzzy, phrase, wildcard, or compound | `$search` + `vectorSearch` operator |
| Pre-filter vector search with simple equality or range | `filter` fields in `$vectorSearch` (see vector-search.md) |
| Cross-collection hybrid search | `$unionWith` + `$vectorSearch` (not `$rankFusion`/`$scoreFusion`) |
**Version requirements**: `$rankFusion` requires MongoDB 8.0+. `$scoreFusion` requires MongoDB 8.2+. Only proceed with this guide if the use case is lexical prefilters, or if the cluster meets the version requirement for the fusion stage of interest. Otherwise do not proceed.
---
## Indexing for Hybrid Search
### For $rankFusion and $scoreFusion
You need two separate indexes on the collection:
**1. A vectorSearch-type index** for the `$vectorSearch` input pipeline:
```javascript
db.collection.createSearchIndex(
"<vector-index-name>",
"vectorSearch",
{
"fields": [
{
"type": "vector",
"path": "<embedding-field>",
"numDimensions": <number>,
"similarity": "dotProduct"
}
]
}
)
```
**2. A search-type index** for the `$search` input pipeline:
```javascript
db.collection.createSearchIndex(
"<search-index-name>",
{
"mappings": { "dynamic": true }
}
)
```
---
### For Lexical Prefilters (vectorSearch Operator)
The `vectorSearch` operator runs inside `$search`, so you need a **single search-type index** that includes a `vector` field type. This is different from a vectorSearch-type index — you cannot use the `$vectorSearch` stage to query fields indexed this way.
```javascript
db.collection.createSearchIndex(
"<search-index-name>",
{
"mappings": {
"dynamic": true,
"fields": {
"<embedding-field>": {
"type": "vector",
"numDimensions": <number>,
"similarity": "dotProduct",
"quantization": "scalar" // Optional
}
}
}
}
)
```
**Note**: `storedSource: true` is not supported on indexes that contain a `vector` field type. Use `include` or `exclude` to specify stored fields explicitly.
---
## Common Rules for Fusion Stages
The following rules apply to both `$rankFusion` and `$scoreFusion`.
**Pipeline naming restrictions**: Pipeline names must not be empty, start with `$`, contain the null character `\0`, or contain `.`
**Not allowed inside input pipelines**: `$project` or `storedSource` fields. Apply modifications (`$project`, `$addFields`, `$set`) in stages after the fusion stage.
---
## $rankFusion
`$rankFusion` executes all input pipelines independently, de-duplicates results, and ranks them using the Reciprocal Rank Fusion (RRF) algorithm. Documents appearing highly ranked in multiple pipelines score highest.
### Syntax
```javascript
{
$rankFusion: {
input: {
pipelines: {
<pipelineName1>: [ <stages> ],
<pipelineName2>: [ <stages> ],
...
}
},
combination: {
weights: {
<pipelineName1>: <number>,
<pipelineName2>: <number>
}
},
scoreDetails: <boolean> // Default: false
}
}
```
### Fields
| Field | Type | Description |
|---|---|---|
| `input.pipelines` | Object | Map of pipeline names to aggregation stages. At least one required. |
| `combination.weights` | Object | Optional. Per-pipeline weights (non-negative numbers). Default weight is 1. |
| `scoreDetails` | Boolean | Optional. If true, populates `$meta: "scoreDetails"` per document. Default false. |
### RRF Formula
For each document, the RRF score is:
```
RRFscore(d) = sum over all pipelines of: weight * (1 / (60 + rank_of_d_in_pipeline))
```
The constant 60 is a sensitivity parameter set by MongoDB and cannot be changed. Documents not present in a pipeline do not contribute a term for that pipeline.
### Input Pipeline Restrictions
See [Common Rules](#common-rules-for-fusion-stages) for naming and modification restrictions. Allowed stages: `$search`, `$vectorSearch`, `$match`, `$geoNear`, `$sample`, `$sort`, `$skip`, `$limit`.
The ordering requirement is satisfied if the pipeline begins with `$search`, `$vectorSearch`, or `$geoNear`, or contains an explicit `$sort`.
---
### Example 1: Basic Hybrid (Vector + Lexical, Equal Weights)
```javascript
db.embedded_movies.aggregate([
{
$rankFusion: {
input: {
pipelines: {
vectorPipeline: [
{
$vectorSearch: {
index: "<vector-index-name>",
path: "plot_embedding",
queryVector: [<query-vector-2048-dimensions>],
numCandidates: 100,
limit: 20
}
}
],
textPipeline: [
{
$search: {
index: "<search-index-name>",
text: {
query: "<query-term>",
path: "title"
}
}
},
{ $limit: 20 }
]
}
}
}
},
{ $limit: 10 }
])
```
**Note**: `$search` does not auto-limit results — always add `$limit` inside the `$search` input pipeline.
---
### Example 2: Weighted Hybrid (Boosting One Pipeline)
Assign higher weight to the pipeline whose ranking should contribute more to the final score:
```javascript
db.embedded_movies.aggregate([
{
$rankFusion: {
input: {
pipelines: {
vectorPipeline: [
{
$vectorSearch: {
index: "<vector-index-name>",
path: "plot_embedding",
queryVector: [<query-vector-2048-dimensions>],
numCandidates: 100,
limit: 20
}
}
],
textPipeline: [
{
$search: {
index: "<search-index-name>",
phrase: {
query: "<query-term>",
path: "title"
}
}
},
{ $limit: 20 }
]
}
},
combination: {
weights: {
vectorPipeline: 0.7,
textPipeline: 0.3
}
}
}
},
{ $limit: 10 }
])
```
**Recommendation**: Set weights per-query based on which method is more appropriate for that query, rather than using static weights for all queries.
---
### Example 3: Multiple $vectorSearch Pipelines
Use multiple vector pipelines to search different fields, different query vectors, or different embedding models:
```javascript
db.embedded_movies.aggregate([
{
$rankFusion: {
input: {
pipelines: {
plotPipeline: [
{
$vectorSearch: {
index: "<vector-index-name>",
path: "plot_embedding_voyage",
queryVector: [<query-vector-2048-dimensions>],
numCandidates: 200,
limit: 50
}
}
],
titlePipeline: [
{
$vectorSearch: {
index: "<vector-index-name>",
path: "title_embedding_voyage",
queryVector: [<query-vector-2048-dimensions>],
numCandidates: 200,
limit: 50
}
}
]
}
},
combination: {
weights: {
plotPipeline: 0.5,
titlePipeline: 0.5
}
}
}
},
{ $limit: 20 }
])
```
---
### Surfacing scoreDetails
Set `scoreDetails: true` on the stage, then project via `$meta: "scoreDetails"`. The output includes a `value` (final RRF score), `description`, and a `details` array — one entry per input pipeline — containing `inputPipelineName`, `rank`, `weight`, and optionally `value` (raw pipeline score). See the `$scoreFusion` scoreDetails section below for a concrete structure example; `$rankFusion` follows the same pattern with `rank` instead of `inputPipelineRawScore`.
---
## $scoreFusion
`$scoreFusion` executes all input pipelines independently, de-duplicates results, and combines them using the actual score values from each pipeline. Supports normalization and custom combination expressions for fine-grained control over how scores are merged.
### Syntax
```javascript
{
$scoreFusion: {
input: {
pipelines: {
<pipelineName1>: [ <stages> ],
<pipelineName2>: [ <stages> ],
...
},
normalization: "none | sigmoid | minMaxScaler"
},
combination: {
weights: {
<pipelineName1>: <number>,
<pipelineName2>: <number>
},
method: "avg | expression",
expression: <arithmetic-expression>
},
scoreDetails: <boolean>
}
}
```
### Fields
| Field | Type | Description |
|---|---|---|
| `input.pipelines` | Object | Map of pipeline names to aggregation stages. At least one required. |
| `input.normalization` | String | Normalize scores before combining: `none` (no normalization), `sigmoid`, or `minMaxScaler`. |
| `combination.weights` | Object | Optional. Per-pipeline weights applied to normalized scores. Default is 1. Mutually exclusive with `combination.expression`. |
| `combination.method` | String | `avg` (default) or `expression`. |
| `combination.expression` | Expression | Custom arithmetic expression. Use pipeline names as variables representing each pipeline's score. Mutually exclusive with `combination.weights`. |
| `scoreDetails` | Boolean | Optional. If true, populates `$meta: "scoreDetails"` per document. Default false. |
### Normalization Options
| Option | Effect |
|---|---|
| `none` | No normalization — raw scores combined as-is |
| `sigmoid` | Applies the sigmoid expression, mapping scores to (0, 1) |
| `minMaxScaler` | Applies the minMaxScaler window operator, scaling scores to [0, 1] |
### Input Pipeline Restrictions
See [Common Rules](#common-rules-for-fusion-stages) for naming and modification restrictions. Allowed stages: `$search`, `$vectorSearch`, `$match`, `$geoNear`, `$sort`, `$skip`, `$limit`. Note: unlike `$rankFusion`, `$sample` is not permitted.
The scoring requirement is satisfied if the pipeline begins with `$search`, `$vectorSearch`, `$match` with legacy text search, or `$geoNear`. Otherwise, include an explicit `$score` stage.
---
### Example 1: avg Method with Weights
```javascript
db.embedded_movies.aggregate([
{
$scoreFusion: {
input: {
pipelines: {
vectorPipeline: [
{
$vectorSearch: {
index: "<vector-index-name>",
path: "plot_embedding",
queryVector: [<query-vector-2048-dimensions>],
numCandidates: 100,
limit: 20
}
}
],
textPipeline: [
{
$search: {
index: "<search-index-name>",
text: {
query: "<query-term>",
path: "title"
}
}
},
{ $limit: 20 }
]
},
normalization: "sigmoid"
},
combination: {
method: "avg",
weights: {
vectorPipeline: 2,
textPipeline: 1
}
}
}
},
{ $limit: 10 }
])
```
---
### Example 2: expression Method with Custom Score Math
Use `expression` when you need full control over how pipeline scores are combined. Reference pipeline names as variables in the expression:
```javascript
db.embedded_movies.aggregate([
{
$scoreFusion: {
input: {
pipelines: {
searchOne: [
{
$vectorSearch: {
index: "<vector-index-name>",
path: "plot_embedding",
queryVector: [<query-vector-2048-dimensions>],
numCandidates: 100,
limit: 20
}
}
],
searchTwo: [
{
$search: {
index: "<search-index-name>",
text: {
query: "<query-term>",
path: "title"
}
}
},
{ $limit: 20 }
]
},
normalization: "sigmoid"
},
combination: {
method: "expression",
expression: {
$sum: [
{ $multiply: ["$searchOne", 10] },
"$searchTwo"
]
}
},
scoreDetails: true
}
},
{
$project: {
_id: 1,
title: 1,
plot: 1,
scoreDetails: { $meta: "scoreDetails" }
}
},
{ $limit: 10 }
])
```
**Note**: `combination.expression` and `combination.weights` are mutually exclusive. When using `expression`, embed weights directly via `$multiply` as shown above.
---
### Surfacing scoreDetails
Set `scoreDetails: true`, then use `$meta: "scoreDetails"` in `$project`, `$addFields`, or `$set`:
```javascript
{
$project: {
title: 1,
scoreDetails: { $meta: "scoreDetails" }
}
}
```
**scoreDetails structure:**
```javascript
{
value: 7.847,
description: "the value calculated by combining the scores...",
normalization: "sigmoid",
combination: {
method: "custom expression",
expression: "{ $sum: [{ $multiply: ['$searchOne', 10] }, '$searchTwo'] }"
},
details: [
{
inputPipelineName: "searchOne",
inputPipelineRawScore: 0.798,
weight: 1,
value: 0.689,
details: []
},
{
inputPipelineName: "searchTwo",
inputPipelineRawScore: 2.962,
weight: 1,
value: 0.950,
details: []
}
]
}
```
---
## Lexical Prefilters (vectorSearch Operator)
The `vectorSearch` operator runs inside a `$search` stage and performs ANN or ENN vector search with the ability to pre-filter using any Atlas Search operator — including `text` with fuzzy matching, `phrase`, `wildcard`, `queryString`, and `compound`. This is more expressive than the MQL-only `filter` option in the `$vectorSearch` stage.
**Requires**: A `search`-type index (not vectorSearch-type) with the embedding field configured as `vector` type. See [Indexing for Hybrid Search](#indexing-for-hybrid-search).
**Cannot be used**: Inside `embeddedDocument`, `compound`, or `facet` operators.
### Syntax
```javascript
{
$search: {
index: "<search-index-name>",
vectorSearch: {
path: "<vector-field>",
queryVector: [<array-of-numbers>],
limit: <number>,
numCandidates: <number>, // Required for ANN (exact: false)
exact: true | false, // Optional, default false
filter: { <search-operator> }, // Optional
score: { <score-options> } // Optional
},
concurrent: true // Optional, dedicated search nodes only
}
}
```
### Key Fields
| Field | Required | Description |
|---|---|---|
| `path` | Yes | The field indexed as `vector` type in the search index |
| `queryVector` | Yes | Array of numbers matching `numDimensions` in the index |
| `limit` | Yes | Number of results to return |
| `numCandidates` | Conditional | Required if `exact` is false or omitted. Max 10000. Recommend 20x `limit`. |
| `exact` | No | `true` for ENN, `false`/omit for ANN |
| `filter` | No | Any Atlas Search operator to pre-filter documents |
| `concurrent` | No | Parallelizes search across segments on dedicated search nodes. Ignored if no dedicated search nodes. |
---
### Example 1: compound Prefilter (queryString + range)
Filter by text match OR date range before running vector search:
```javascript
db.embedded_movies.aggregate([
{
$search: {
index: "<search-index-name>",
vectorSearch: {
path: "plot_embedding",
queryVector: [<query-vector-2048-dimensions>],
limit: 10,
exact: true,
filter: {
compound: {
should: [
{
queryString: {
defaultPath: "fullplot",
query: "plot:courtroom OR lawyer"
}
},
{
range: {
path: "year",
gte: 2000,
lte: 2015
}
}
]
}
}
},
concurrent: true
}
},
{
$project: {
_id: 0,
title: 1,
plot: 1,
score: { $meta: "searchScore" }
}
}
])
```
---
### Example 2: text Prefilter with Fuzzy Matching
Filter by fuzzy text match before running ANN vector search:
```javascript
db.embedded_movies.aggregate([
{
$search: {
index: "<search-index-name>",
vectorSearch: {
path: "plot_embedding",
queryVector: [<query-vector-2048-dimensions>],
limit: 10,
numCandidates: 200,
filter: {
text: {
path: "fullplot",
query: "charming animal",
fuzzy: {}
}
}
},
concurrent: true
}
},
{
$project: {
_id: 0,
title: 1,
plot: 1,
score: { $meta: "searchScore" }
}
}
])
```
---
## Best Practices and Limitations
### Best Practices
**Set limits inside $search sub-pipelines**: `$search` does not limit results by default. Always add `$limit` inside the input pipeline, or `$rankFusion`/`$scoreFusion` evaluates all search results.
```javascript
textPipeline: [
{ $search: { ... } },
{ $limit: 20 } // Required
]
```
**Set weights per-query**: Tune weights based on which search method is most appropriate for a given query rather than using fixed weights for all queries. This improves relevance and resource utilization.
**Handle disjoint results**: If most results come from one pipeline and not the other, the two methods are returning largely different documents. Increase per-pipeline limits to improve overlap.
**Use `$match` for non-search filtering**: To filter on specific fields without a search pipeline (e.g., boost on a flag field), add a `$match` pipeline inside `input.pipelines`. It must contain an explicit `$sort` to qualify as a ranked pipeline.
### Limitations
**Single collection only**: `$rankFusion` and `$scoreFusion` cannot span multiple collections. For cross-collection hybrid search, use `$unionWith` with `$vectorSearch`.
**Pipelines run serially**: Input pipelines do not execute in parallel.
**No pagination inside sub-pipelines**: `$rankFusion` and `$scoreFusion` do not support pagination within input pipelines.
**vectorSearch operator restrictions**: Cannot be used inside `embeddedDocument`, `compound`, or `facet` operators. Cannot use `highlight`, `sort`, or `searchSequenceToken` with the `vectorSearch` operator — use `$skip` and `$limit` after `$search` instead.
references/lexical-search-indexing.md
# Lexical Search - Indexing
This guide covers how to configure MongoDB Atlas Search indexes. Use this reference to build index definitions with proper field types, analyzers, mappings, and optimization settings.
## Table of Contents
- [Atlas Search Index Definition](#atlas-search-index-definition)
- [Analyzer Selection](#analyzer-selection)
- [Field Types](#field-types)
- [Dynamic vs Explicit Mappings](#dynamic-vs-explicit-mappings)
- [Stored Source](#stored-source)
- [Synonyms](#synonyms)
---
## Atlas Search Index Definition
### Syntax
```javascript
{
"analyzer": "<analyzer-for-index>",
"searchAnalyzer": "<analyzer-for-query>",
"mappings": {
"dynamic": <boolean> | {
"typeSet": "<typeSet-name>"
},
"fields": {
<field-definition>
}
},
"numPartitions": <integer>,
"analyzers": [ <custom-analyzer> ],
"storedSource": <boolean> | {
<stored-source-definition>
},
"synonyms": [
{
<synonym-mapping-definition>
}
],
"typeSets": [
{
"types": [
{<field-types-definition>}
]
}
]
}
```
### Options
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `analyzer` | String | Optional | Specifies the analyzer to apply to string fields when indexing. If set only at the top level and not specified for individual fields, applies to all fields. If omitted, defaults to Standard Analyzer. |
| `searchAnalyzer` | String | Optional | Specifies the analyzer to apply to query text before searching. If omitted, defaults to the `analyzer` option. If both omitted, defaults to Standard Analyzer. |
| `mappings` | Object | Required | Specifies how to index fields at different paths for this index. |
| `mappings.dynamic` | Boolean or Object | Optional | Enables dynamic mapping of field types or configures fields individually. Set to `true` to recursively index all indexable field types, `false` to only index fields specified in `mappings.fields`, or specify a `typeSet` for configurable dynamic indexing. If omitted, defaults to `false`. **Note:** Dynamic indexing automatically and recursively indexes all nested documents unless explicitly disabled. |
| `mappings.dynamic.typeSet` | String | Optional | References the name of the `typeSets` object that contains the list of field types to automatically and recursively index. Mutually exclusive with `mappings.dynamic` boolean flag. |
| `mappings.fields` | Object | Conditional | Specifies the fields that you want to index. Required only if `dynamic` is `false`. You can't index fields that contain the dollar ($) sign at the start of the field name. |
| `numPartitions` | Integer | Optional | Specifies the number of sub-indexes to create if the document count exceeds two billion. Valid values: 1, 2, 4. If omitted, defaults to 1. Requires search nodes deployed in your cluster. |
| `analyzers` | Array of Custom Analyzers | Optional | Specifies the custom analyzers to use in this index. Reference by name in `analyzer`, `searchAnalyzer`, or field-level analyzer options. |
| `storedSource` | Boolean or Object | Optional | Specifies fields in documents to store for query-time look-ups using `returnStoredSource`. Can be `true` (store all fields), `false` (store no fields), or an object specifying fields to include/exclude. Available on clusters running MongoDB 7.0+. If omitted, defaults to `false`. |
| `synonyms` | Array of Synonym Mapping Definition | Optional | Specifies synonym mappings to use in your index. An index definition can have only one synonym mapping. |
| `typeSets` | Array of Objects | Optional | Specifies the typeSets to use for dynamic mappings. |
| `typeSets.[n].name` | String | Required | Specifies the name of the typeSet configuration. |
| `typeSets.[n].types` | Array of Objects | Required | Specifies the field types to index automatically using dynamic mappings. |
| `typeSets.[n].types.[n].type` | String | Required | Specifies the field type to automatically index (e.g., "string", "number", "date"). |
### Basic Definition
Most indexes only need the mappings configuration:
```javascript
{
"mappings": {
"dynamic": <boolean> | { <typeSet-definition> },
"fields": { <field-definition> }
}
}
```
---
## Analyzer Selection
The analyzer determines how text is processed for indexing and searching.
**Default behavior:** Most queries don't specify an analyzer and use MongoDB's default **standard analyzer**, which:
- Divides text into terms based on word boundaries (language-neutral)
- Converts terms to lowercase and removes punctuation
- Recognizes email addresses, acronyms, CJK characters, alphanumerics, and more
**Common built-in analyzers:**
| Analyzer | Use Case | Example |
|----------|----------|---------|
| `lucene.standard` | General text search (default) | "The quick brown fox" → ["quick", "brown", "fox"] |
| `lucene.simple` | Lowercase, no special chars | "Hello-World!" → ["hello", "world"] |
| `lucene.keyword` | Exact matching, facets | "Action" → ["Action"] |
| `lucene.whitespace` | Split on spaces only | "first-class" → ["first-class"] |
| Language-specific | Stemming, stop words | `lucene.english`, `lucene.spanish` |
---
### Index Analyzer (Applied at Index Time)
Specify per-field or top-level for all fields:
```javascript
// Field-level (add to mappings.fields):
{
"title": {
"type": "string",
"analyzer": "lucene.standard" // Or omit to use default
},
"category": {
"type": "token" // Exact matching — use token, not string with lucene.keyword
}
}
// Top-level (applies to all fields unless overridden):
{
"analyzer": "lucene.standard",
"mappings": {
"fields": {
"title": { "type": "string" } // Uses top-level analyzer
}
}
}
```
---
### Search Analyzer (Applied at Query Time)
Apply different analysis to queries than to indexed content:
```javascript
// Top-level searchAnalyzer:
{
"searchAnalyzer": "lucene.simple", // Query-time analyzer
"mappings": {
"fields": {
"description": {
"type": "string",
"analyzer": "lucene.standard" // Index-time analyzer
}
}
}
}
```
**Use case:** Index with standard analysis, but search with simpler/synonym-aware analyzer.
If omitted, uses the index `analyzer`. If both omitted, defaults to `lucene.standard`.
---
### Multi Analyzer (Alternate Analyzers for Same Field)
Index the same field with multiple analyzers:
```javascript
// Field configuration (add to mappings.fields):
{
"title": {
"type": "string",
"analyzer": "lucene.standard", // Default analyzer
"multi": {
"keywordAnalyzer": {
"type": "string",
"analyzer": "lucene.keyword" // Alternate analyzer
}
}
}
}
```
**Use case:** Support both fuzzy matching and exact matching on the same field.
To query using the alternate analyzer, specify the path as `fieldName.alternateAnalyzerName` (e.g., `title.keywordAnalyzer`).
---
### Custom Analyzers
Define custom tokenization and filtering:
```javascript
// Index definition with custom analyzer:
{
"analyzers": [
{
"name": "customAnalyzer",
"tokenizer": {
"type": "standard"
},
"charFilters": [],
"tokenFilters": [
{ "type": "lowercase" },
{ "type": "stop", "tokens": ["the", "a", "an"] }
]
}
],
"mappings": {
"fields": {
"content": {
"type": "string",
"analyzer": "customAnalyzer" // Reference custom analyzer
}
}
}
}
```
**Use case:** Need specific tokenization or filtering not provided by built-in analyzers.
---
### Normalizers (Token Type Only)
Normalizers produce a single token (used with `token` field type):
```javascript
// Field configuration (add to mappings.fields):
{
"username": {
"type": "token",
"normalizer": "lowercase" // Options: "lowercase", "none"
}
}
```
**Normalizers:**
- `lowercase`: Transforms to lowercase, creates single token
- `none`: No transformation, creates single token
**Use case:** Exact matching with case normalization for token fields.
---
### Decision Guide
- **lucene.standard** (or omit): Default for most text fields
- **lucene.keyword**: Categories, tags
- **Language-specific**: When you know the content language
- **searchAnalyzer**: Different analysis for queries vs indexed content (e.g., synonyms)
- **multi**: Support multiple search patterns on same field
- **Custom**: Need specific tokenization/filtering logic
- **Normalizers**: Token fields requiring case normalization
---
## Field Types
**Dynamic mapping includes:** boolean, date, number, objectId, string, uuid
**Must configure explicitly:** autocomplete, token, geo, embeddedDocuments, vector
### Quick Reference
| Type | When to Use | Required Fields | Optional Fields (with valid values) | Notes |
|------|-------------|-----------------|-------------------------------------|-------|
| **string** | Full-text search, phrase matching, fuzzy search | type | analyzer, searchAnalyzer, indexOptions: "docs" or "freqs" or "positions" or "offsets", store: true or false, multi | Default for text. For sorting use token instead. |
| **token** | Sort/facet on text, exact matching | type | normalizer: "lowercase" or "none" (default: "none") | Required for sorting or faceting strings. Max 8181 chars. |
| **autocomplete** | Search-as-you-type, typeahead, partial or substring matching | type | analyzer, tokenization: "edgeGram" or "rightEdgeGram" or "nGram" (default: "edgeGram"), minGrams (default: 2), maxGrams (default: 15), foldDiacritics: true or false | Not included in "dynamic: true". Recommend maxGrams ≤ 15. |
| **boolean** | True/false filters | type | None | Included in "dynamic: true". |
| **date** | Date ranges, timestamps | type | None | Included in "dynamic: true". |
| **number** | Numeric queries, ranges, sorting | type | representation: "int64" or "double" (default: "double"), indexIntegers: true or false, indexDoubles: true or false | Included in "dynamic: true". Use int64 for large integers. |
| **objectId** | Query by _id | type | None | Included in "dynamic: true". Standard MongoDB ObjectIds. |
| **uuid** | UUID identifiers | type | None | Included in "dynamic: true". BSON Binary Subtype 4. |
| **geo** | Location search, geographic queries | type | indexShapes: true or false (default: false) | Included in "dynamic: true". Requires GeoJSON. Set indexShapes=true for polygons. |
| **embeddedDocuments** | Search in arrays of objects, independent scoring | type | dynamic: true or false or {typeSet: "name"} (default: false), fields, storedSource: true or false or {include/exclude} | Not included in "dynamic: true". Max 5 nesting levels. Each nested document counts toward 2.1B limit. |
| **vector** | Lexical prefilters for semantic search | type, numDimensions (1-8192), similarity: "cosine" or "dotProduct" or "euclidean" | quantization: "none" or "scalar" or "binary" (default: "none"), hnswOptions.maxEdges: 16-64, hnswOptions.numEdgeCandidates: 100-3200 | Not included in "dynamic: true". For hybrid search. See vector-search.md and hybrid-search.md. |
**Field definition structure:**
```javascript
{
"mappings": {
"fields": {
"<field-name>": {
"type": "<field-type>",
// type-specific options here
}
}
}
}
```
**Multiple types on same field:**
```javascript
"<field-name>": [
{ "type": "string" },
{ "type": "token", "normalizer": "lowercase" }
]
```
**Arrays:** MongoDB Search automatically flattens arrays during indexing. Specify only the element type, not that it's an array.
---
## Dynamic vs Explicit Mappings
**Choose based on user's needs:**
**Use dynamic (true)** when:
- User is prototyping or exploring data with unknown schema
- Need to get started quickly without defining all fields
- All or most fields need to be searchable
- Accept larger index size and slower performance for convenience
**Use explicit (dynamic: false)** (recommended for production) when:
- User has completed early stages of prototyping and knows exactly which fields to search
- Performance and index size are priorities
- Schema is stable and well-defined
- Only a subset of fields need to be searchable
**Use typeSets (recommended for production)** when:
- User wants automatic indexing but with control over which types
- Document schema is dynamic and new fields need to be indexed automatically without an index rebuild
- Want different indexing strategies for different nested documents
- Balance between convenience and performance is important
---
**Dynamic mappings** automatically index all fields:
```javascript
{
"mappings": {
"dynamic": true // Index everything
}
}
```
- **Pros**: Quick setup, works immediately
- **Cons**: Larger index, slower queries, wastes resources on unused fields
**Explicit mappings** define exactly what to index:
```javascript
{
"mappings": {
"dynamic": false, // Only index specified fields
"fields": {
"title": { "type": "string" },
"genre": { "type": "token" }
}
}
}
```
- **Pros**: Smaller index, faster queries, precise control
- **Cons**: Requires knowing your schema
**Configurable dynamic with typeSets** (recommended middle ground):
```javascript
{
"mappings": {
"dynamic": {
"typeSet": "customTypes"
},
"fields": {
"metadata": {
"type": "document",
"dynamic": {
"typeSet": "metadataTypes"
}
}
}
},
"typeSets": [
{
"name": "customTypes",
"types": [
{ "type": "string" },
{ "type": "number" }
]
},
{
"name": "metadataTypes",
"types": [
{
"type": "string",
"analyzer": "lucene.standard"
}
]
}
]
}
```
- **Pros**: Automatically indexes specified field types, more control than full dynamic, can configure different typeSets for sub-documents
- **Cons**: Still indexes all fields of specified types
**Recommendation:** Use static mappings or a dynamic typeSet (within a specific path, not at the root document level) in production for optimized index size and performance.
---
## Stored Source
Store frequently accessed fields directly in the search index (mongot) to avoid full document lookups from the database. This dramatically improves query performance, especially when filtering or sorting.
**Requirements:**
- Available on clusters running MongoDB 7.0+
- Stored fields must still be indexed separately to query them
- Retrieve stored fields at query time using returnStoredSource: true (see lexical-search-querying.md)
**Syntax:**
```javascript
{
"storedSource": true | false | {
"include" | "exclude": ["<field-name>", ...]
}
}
```
**Options:**
true - Store all fields in documents. Not supported if index contains vector type field. Can significantly impact performance.
false - Don't store any fields (default behavior).
{ "include": [...] } - Store only specified fields. MongoDB Search also stores _id by default. List field names or dot-separated paths.
{ "exclude": [...] } - Store all fields except specified ones.
**Examples:**
**Store specific fields:**
```javascript
{
"mappings": {
"dynamic": false,
"fields": {
"title": { "type": "string" },
"genre": { "type": "token" },
"year": { "type": "number" },
"rating": { "type": "number" }
}
},
"storedSource": {
"include": ["title", "genre", "year", "rating"]
}
}
```
**Exclude specific fields:**
```javascript
{
"storedSource": {
"exclude": ["largeTextField", "unusedField"]
}
}
```
**Store all fields:**
```javascript
{
"storedSource": true
}
```
**When to use:**
- Fields used for filtering, sorting, or projection after $search
- Frequently accessed fields in search results
- When avoiding database lookups is critical for performance
**When NOT to use:**
- Very large text fields (increases index size significantly)
- Fields rarely used in queries
- When index size is a concern
**Note:** For using stored source at query time, see lexical-search-querying.md. For vector field storage considerations, see vector-search.md.
---
## Synonyms
Use when user wants query expansion with equivalent terms (e.g., "car" also finds "automobile", "vehicle").
**Agent Workflow:**
1. **Ask user to create synonym collection** in the same database as their indexed collection.
2. **Provide synonym document format** based on user's needs:
**For bidirectional synonyms** (all terms interchangeable):
```javascript
db.synonyms.insertMany([
{
"mappingType": "equivalent",
"synonyms": ["car", "vehicle", "automobile"]
},
{
"mappingType": "equivalent",
"synonyms": ["happy", "joyful", "glad"]
}
])
```
**For one-way synonyms** (input maps to synonyms only):
```javascript
db.synonyms.insertMany([
{
"mappingType": "explicit",
"input": ["pants"],
"synonyms": ["trousers", "slacks"]
}
])
```
3. **Add synonym mapping to index definition:**
```javascript
{
"mappings": {
"fields": {
"<field-name>": {
"type": "string",
"analyzer": "lucene.standard" // Note the analyzer
}
}
},
"synonyms": [
{
"name": "<synonym-mapping-name>",
"analyzer": "lucene.standard", // Must match field analyzer
"source": {
"collection": "<synonym-collection-name>"
}
}
]
}
```
**Critical Rules:**
- Synonym mapping analyzer MUST match the field analyzer being queried
- Only one synonym mapping allowed per index
- Changes to synonym collection auto-update (no reindex needed)
- Works only with text and phrase operators
**Example:**
If user wants "car" to also find "vehicle" and "automobile":
1. Tell user to create collection: db.synonyms.insertOne({ "mappingType": "equivalent", "synonyms": ["car", "vehicle", "automobile"] })
2. Add to index with analyzer matching the field being searched
3. Queries automatically expand (user searches "car", MongoDB Search searches: car OR vehicle OR automobile)
---
## Searching on Views
**Requires MongoDB 8.0+.** Create Atlas Search indexes on Views to partially index a collection, transform documents, or support incompatible data types.
**Note**: Programmatic index creation via `mongosh`/driver methods requires **8.1+**. On 8.0, also note that queries must run against the **source collection** referencing the view's index name. On 8.1+, you can query the view directly.
**Supported view stages**: `$addFields`, `$set`, `$match` with `$expr` only.
**Key limitations**:
- Index names must be unique across source collection and all its views
- No operators producing dynamic results (e.g., `$USER_ROLES`, `$rand`)
- Queries return original source documents. Use `storedSource` to retrieve transformed fields
**Example: partial index (filter documents)**
```javascript
db.createView("movies_After2000", "movies", [
{ $match: { $expr: { $gt: ["$released", ISODate("2000-01-01")] } } }
])
db.movies_After2000.createSearchIndex(
"after2000Index",
{ "mappings": { "dynamic": true } }
)
// 8.1+: query view directly; 8.0: query source collection using index name
db.movies_After2000.aggregate([
{ $search: { index: "after2000Index", text: { path: "title", query: "<query>" } } }
])
```
**Editing a view**: Use `collMod`. MongoDB Search auto-reindexes on view definition changes with no downtime.
**Performance**: Complex transformations slow performance. For heavy transformations consider a materialized view, or query the source collection directly.
**Troubleshooting**:
- Index goes **FAILED**: view is incompatible with Search, or source collection was removed/changed
- Index goes **STALE**: view's pipeline fails on a document. Index remains queryable while STALE; returns to READY after fixing the document or view definition
- **`$search is only valid as first stage`** error: you're on MongoDB 8.0 querying the view directly. Query the source collection instead, or upgrade to 8.1+
references/lexical-search-querying.md
# Lexical Search - Querying
This guide covers query patterns and optimization techniques for MongoDB Atlas Search.
## Table of Contents
- [$search vs $searchMeta](#search-vs-searchmeta)
- [Query Patterns](#query-patterns)
- [Query Optimization](#query-optimization)
- [Query Performance Analysis](#query-performance-analysis)
---
## $search vs $searchMeta
Both stages must be the **first stage** in an aggregation pipeline.
| Stage | Use When |
|---|---|
| `$search` | You need matching documents, with or without metadata |
| `$searchMeta` | You only need metadata (count, facets) — no documents returned |
`$searchMeta` shares the following fields with `$search`: `index`, all operator names (e.g. `text`, `range`, `compound`), `concurrent` (parallelizes search across segments on dedicated search nodes only — ignored otherwise), and `returnStoredSource`.
---
## Query Patterns
### Operator Reference
| Operator | Description |
|---|---|
| `autocomplete` | Search-as-you-type from incomplete input |
| `compound` | Combines multiple operators into a single query |
| `embeddedDocument` | Queries fields inside arrays of objects |
| `equals` | Exact match on boolean, date, number, objectId, token, uuid |
| `exists` | Tests for presence of a field |
| `geoShape` | Queries shapes by spatial relation (geo type, indexShapes: true) |
| `geoWithin` | Queries points within a region (geo type) |
| `hasAncestor` | Queries ancestor-level fields when using `returnScope` |
| `hasRoot` | Queries root-level fields when using `returnScope` |
| `in` | Queries single values or arrays of values |
| `moreLikeThis` | Finds documents similar to a given document |
| `near` | Queries values near a number, date, or geo point |
| `phrase` | Searches for terms in a specific order |
| `queryString` | Boolean/field-specific query syntax |
| `range` | Queries values within a numeric, date, string, or objectId range |
| `regex` | Regular expression matching on string fields |
| `text` | Full-text analyzed search on string fields |
| `vectorSearch` | Semantic search with lexical pre-filters (vector type in search index) |
| `wildcard` | Wildcard pattern matching on string fields |
---
### Count Results
Use the `count` option in `$searchMeta` to count matching documents without fetching them. Also works in `$search` via the `$SEARCH_META` aggregation variable when you need both results and count.
```javascript
// Count only (recommended)
db.movies.aggregate([
{
$searchMeta: {
range: { path: "year", gte: 2010, lte: 2015 },
count: { type: "lowerBound" } // or "total" for exact count
}
}
])
// Returns: { count: { lowerBound: NumberLong(1001) } }
```
```javascript
// Count alongside results using $SEARCH_META
db.movies.aggregate([
{
$search: {
text: { path: "title", query: "<query>" },
count: { type: "total" }
}
},
{ $project: { title: 1, meta: "$SEARCH_META" } },
{ $limit: 10 }
])
```
| type | Behavior |
|---|---|
| `lowerBound` | Approximate. Exact up to `threshold` (default 1000), rough above it. |
| `total` | Exact count. Slower on large result sets. |
**Note:** Count affects performance — use only when needed (e.g., first page of paginated results).
---
### Pagination with searchSequenceToken
Cursor-based pagination using tokens. More efficient than `$skip` alone for deep pagination.
**Step 1 — Get tokens from the initial query:**
```javascript
db.movies.aggregate([
{
$search: {
index: "<index-name>",
text: { path: "title", query: "summer" },
sort: { released: 1, _id: 1 } // Sort on a unique field to prevent tie-ordering issues
}
},
{ $limit: 10 },
{
$project: {
title: 1, released: 1,
paginationToken: { $meta: "searchSequenceToken" }
}
}
])
```
**Step 2 — Next page using searchAfter:**
```javascript
db.movies.aggregate([
{
$search: {
index: "<index-name>",
text: { path: "title", query: "summer" },
searchAfter: "<token-from-last-document-on-previous-page>",
sort: { released: 1, _id: 1 } // maintain the same sort order
}
},
{ $limit: 10 },
{ $project: { title: 1, paginationToken: { $meta: "searchSequenceToken" } } }
])
```
Use `searchBefore` with the first document's token on the current page to go to the previous page — results are returned in reverse order. Combine `searchAfter` with `$skip` to jump pages.
**Key constraint:** Query semantics (operator, path, query value, sort) must be identical between the initial query and any `searchAfter`/`searchBefore` query.
---
### Retrieve Arrays of Objects with returnScope
Return each element of an embedded document array as an individually scored document. Works in both `$search` and `$searchMeta`.
**Requirements:**
- Array field indexed as `embeddedDocuments` type with `storedSource` defined on the fields to return
- `returnStoredSource: true` in the query
- All operator paths must be nested under `returnScope.path` (use `hasAncestor` or `hasRoot` to query outside it)
**Index:**
```javascript
{
"mappings": {
"dynamic": false,
"fields": {
"funding_rounds": {
"type": "embeddedDocuments",
"dynamic": true,
"storedSource": {
"include": ["round_code", "raised_currency_code", "raised_amount"]
}
}
}
}
}
```
**Query:**
```javascript
db.companies.aggregate([
{
$search: {
range: { path: "funding_rounds.raised_amount", gte: 5000000, lte: 10000000 },
returnStoredSource: true,
returnScope: { path: "funding_rounds" }
}
},
{ $limit: 5 }
])
```
Only fields defined in `storedSource` within the embedded document are returned — root-level fields are excluded. When `returnScope` is specified, all query paths must start with `returnScope.path`.
---
### Advanced Query Syntax (queryString)
**Use case:** Complex search with boolean operators, wildcards, and field-specific queries.
**Fields configuration:**
```javascript
// Add to mappings.fields in your index:
{
"title": { "type": "string" },
"director": { "type": "string" },
"year": { "type": "number" }
}
```
**Query patterns:**
```javascript
// Boolean operators
db.collection.aggregate([
{
$search: {
index: "search_index",
queryString: {
defaultPath: "title",
query: "detective AND (noir OR thriller) NOT comedy"
}
}
}
])
// Field-specific searches
db.collection.aggregate([
{
$search: {
index: "search_index",
queryString: {
defaultPath: "title",
query: "title:inception AND director:nolan"
}
}
}
])
// Wildcards and ranges
db.collection.aggregate([
{
$search: {
index: "search_index",
queryString: {
defaultPath: "title",
query: "star* AND year:[2010 TO 2020]"
}
}
}
])
```
**Supported syntax:**
- Boolean: `AND`, `OR`, `NOT`
- Grouping: `(term1 OR term2)`
- Wildcards: `*` (0+ chars), `?` (single char)
- Ranges: `[min TO max]` for numbers/dates
- Field-specific: `fieldName:value`
**Key considerations:**
- Great for building search UIs with advanced options
- Users can construct complex queries without API changes
- Validate/sanitize user input to prevent injection
---
### Searching Nested Arrays (embeddedDocument)
**Use case:** Search within arrays of objects where element-wise comparisons are required (similar to $elemMatch), or each element must be scored independently.
**Fields configuration:**
```javascript
// Add to mappings.fields in your index:
{
"title": { "type": "string" },
"reviews": {
"type": "embeddedDocuments", // Required for array search
"fields": {
"author": { "type": "string" },
"text": { "type": "string" },
"rating": { "type": "number" }
}
}
}
```
**Query pattern:**
```javascript
db.collection.aggregate([
{
$search: {
index: "search_index",
embeddedDocument: {
path: "reviews",
operator: {
compound: {
must: [
{ text: { query: "excellent", path: "reviews.text" } }
],
filter: [
{ range: { path: "reviews.rating", gte: 4 } }
]
}
},
score: { embedded: { aggregate: "maximum" } } // or sum, minimum, mean
}
}
}
])
```
**Score aggregation options:**
- `sum`: Add scores from all matching array elements
- `maximum`: Use highest score from array elements
- `minimum`: Use lowest score from array elements
- `mean`: Average scores from array elements
**Key considerations:**
- Each array element is indexed as a separate document
- Use `embeddedDocuments` field type, not regular `document`
- Score aggregation controls how array matches affect overall document score
- Performance can be degraded due to complexity of parent-child joins
---
### Search Highlighting
**Use case:** Show users which parts of documents matched their query.
**Fields configuration:**
```javascript
// Add to mappings.fields in your index:
{
"title": { "type": "string" },
"plot": { "type": "string" }
}
```
**Query pattern:**
```javascript
db.collection.aggregate([
{
$search: {
index: "search_index",
text: {
query: "detective noir",
path: "plot"
},
highlight: {
path: "plot",
maxCharsToExamine: 500000, // Default
maxNumPassages: 5 // Number of snippets
}
}
},
{
$project: {
title: 1,
plot: 1,
highlights: { $meta: "searchHighlights" },
score: { $meta: "searchScore" }
}
}
])
```
**Highlight result structure:**
```javascript
{
"highlights": [
{
"path": "plot",
"texts": [
{ "value": "A ", "type": "text" },
{ "value": "detective", "type": "hit" },
{ "value": " investigates a murder in ", "type": "text" },
{ "value": "noir", "type": "hit" },
{ "value": " Los Angeles", "type": "text" }
],
"score": 1.23
}
]
}
```
**Key considerations:**
- `type: "hit"` indicates matched terms
- `type: "text"` is surrounding context
- Multiple passages returned for long documents
- Use in search results UI to show match context
---
### Compound Queries
**Compound queries** combine multiple operators efficiently:
```javascript
db.collection.aggregate([
{
$search: {
index: "search_index",
compound: {
must: [
{ text: { query: "detective", path: "plot" } } // Required, affects score
],
should: [
{ text: { query: "mystery", path: "genre" } } // Optional, boosts score
],
filter: [
{ range: { path: "year", gte: 2000 } } // Required, no score impact
],
mustNot: [
{ text: { query: "comedy", path: "genre" } } // Excludes results
]
}
}
}
])
```
**Clause types:**
- `must`: Required matches that affect scoring
- `should`: Optional matches that boost scores
- `filter`: Required matches that don't affect scoring (faster)
- `mustNot`: Exclusions
**Performance tips:**
- Use `filter` instead of `must` for criteria that shouldn't affect scoring (faster)
- Put most selective criteria in `must` or `filter` first
- Limit `should` clauses to 3-5 for best performance
---
### Query with Synonyms
When your index is configured with synonyms, specify the synonym mapping name in your query:
```javascript
db.collection.aggregate([
{
$search: {
index: "search_index",
text: {
query: "car chase",
path: "description",
synonyms: "synonym-mapping-name" // Reference the mapping from your index
}
}
}
])
```
**Note:** When you specify a synonym mapping name, MongoDB Search automatically searches for the query terms AND all their synonyms (e.g., "car" also matches "automobile", "vehicle").
---
### Using Multi Analyzers
Query specific analyzer variants of a field:
```javascript
// Standard fuzzy search
db.collection.aggregate([
{
$search: {
index: "search_index",
text: {
query: "Action",
path: "title" // Uses default analyzer
}
}
}
])
// Exact match using keyword analyzer
db.collection.aggregate([
{
$search: {
index: "search_index",
text: {
query: "Action",
path: "title.keywordAnalyzer" // Uses alternate analyzer
}
}
}
])
```
**Use case:** Support both fuzzy and exact matching on the same field without duplicating data.
---
### Autocomplete
Search-as-you-type on fields indexed as `autocomplete` type (see lexical-search-indexing.md).
| Option | Description |
|---|---|
| `query` | String to search |
| `path` | Field indexed as `autocomplete` |
| `tokenOrder` | `any` (tokens in any order; sequential matches score higher) or `sequential` (tokens must be adjacent) |
| `fuzzy` | `{ maxEdits: 1\|2, prefixLength: <n>, maxExpansions: <n> }` |
To score exact matches higher, index the field as both `autocomplete` and `string` types and query using `compound`.
---
### Facet
Groups results into buckets by field values or ranges. Use with `$searchMeta` for metadata only, or with `$search` + `$SEARCH_META` variable for results and metadata.
```javascript
{ "$searchMeta": { "facet": {
"operator": { <operator> },
"facets": {
"<facet-name>": { "type": "string|number|date", "path": "<field>", ...options }
}
} } }
```
| Facet type | Field index type | Bucket definition |
|---|---|---|
| `string` | `token` | Top N unique string values. `numBuckets` defaults to 10. |
| `number` | `number` | Numeric ranges via `boundaries` array + optional `default` bucket |
| `date` | `date` | Date ranges via `boundaries` array + optional `default` bucket |
---
### geoShape
Query shapes by spatial relation. Field must be indexed as `geo` type with `indexShapes: true`. Required fields: `geometry` (GeoJSON Polygon, MultiPolygon, or LineString), `path`, and `relation`:
| relation | Meaning |
|---|---|
| `contains` | Indexed geometry contains the query geometry |
| `disjoint` | No overlap between geometries |
| `intersects` | Geometries overlap |
| `within` | Indexed geometry is within the query geometry (not supported for LineString or Point) |
---
### geoWithin
Query geographic points within a region. Field must be indexed as `geo` type. Specify one of:
- `box`: `{ bottomLeft: <GeoJSON Point>, topRight: <GeoJSON Point> }`
- `circle`: `{ center: <GeoJSON Point>, radius: <meters> }`
- `geometry`: GeoJSON Polygon or MultiPolygon
**For both geo operators:** longitude must be specified before latitude; longitude range [-180, 180], latitude range [-90, 90].
---
## Query Optimization
### Sorting Search Results
Use the `sort` option inside `$search` to sort at the mongot level (more efficient than a `$sort` stage after). Supports: `boolean`, `date`, `number`, `objectId`, `uuid`, and `string` (must be indexed as `token` type). Cannot sort on `embeddedDocuments` type fields.
```javascript
db.collection.aggregate([
{
$search: {
text: { ... },
sort: { "fieldName": -1, "title": 1, score: { $meta: "searchScore" } }
}
},
{ $limit: 10 }
])
```
**Sort by score:**
```javascript
sort: { score: { $meta: "searchScore", order: 1 } } // ascending (lowest score first)
sort: { score: { $meta: "searchScore" } } // descending (default)
```
**Null/missing values:** Appear first in ascending sort by default. Use `noData: "highest"` to push them last:
```javascript
sort: { "field": { order: 1, noData: "highest" } }
```
**Key rules:**
- `sort` inside `$search` only works on indexed fields — use `$sort` after for non-indexed or computed fields
- For `searchSequenceToken` pagination, sort must include a unique field (e.g., `_id`) to avoid tie-ordering
- Arrays: ascending uses smallest element, descending uses largest
### Using Stored Source
Retrieve frequently accessed fields directly from the search index instead of the database:
```javascript
db.collection.aggregate([
{
$search: {
index: "search_index",
text: { query: "detective", path: "plot" },
returnStoredSource: true // Retrieve from mongot, not DB
}
},
{ $limit: 20 },
{ $match: { rating: { $gte: 7 } } } // Filter on stored fields
])
```
**Requirements:**
- Fields must be configured in `storedSource` in your index definition
- Dramatically improves performance by avoiding database lookups
- Especially beneficial when filtering or sorting after $search
---
### $match After $search
Minimize blocking stages after `$search` — prefer encapsulating filter logic inside the `$search` stage itself using `compound.filter`. This avoids additional mongod operations and makes full use of the Atlas Search index.
**Prefer `compound.filter` over `$match`** for fields indexed in the search index (string, token, number, date, boolean, objectId, uuid, geo):
```javascript
// Prefer this
{ $search: { compound: { must: [{ text: { ... } }], filter: [{ range: { path: "year", gte: 2000 } }] } } }
// Avoid this where possible
{ $search: { text: { ... } } },
{ $match: { year: { $gte: 2000 } } }
```
**If you must use `$match`** (e.g., for non-indexed or computed fields), use `storedSource` + `returnStoredSource` to avoid a full document lookup in mongod:
```javascript
{ $search: { text: { ... }, returnStoredSource: true } },
{ $match: { storedField: { $exists: true } } }
```
---
## Query Performance Analysis
Use `explain` to analyze query performance:
```javascript
db.collection.explain("executionStats").aggregate([
{ $search: { /* ... */ } }
])
```
**Important:** Atlas Search explain output differs from standard MongoDB explain. It shows execution on the search engine (mongot) side with Lucene-specific statistics, not standard MongoDB execution plans.
references/vector-search.md
# Vector Search - Indexing and Querying
This guide covers how to configure MongoDB Vector Search indexes and construct queries for semantic similarity search.
**Scope**: This guide covers pure vector search indexes. For hybrid search (combining lexical and vector search), see hybrid-search.md.
## Table of Contents
- [Vector Search Index Definition](#vector-search-index-definition)
- [Index Configuration Parameters](#index-configuration-parameters)
- [Filter Fields (Pre-filtering)](#filter-fields-pre-filtering)
- [Query Construction](#query-construction)
- [Query Optimization](#query-optimization)
---
## Vector Search Index Definition
### Syntax
MongoDB Vector Search index definitions have the following structure:
```javascript
{
"fields": [
{
"type": "vector",
"path": "<field-to-index>",
"numDimensions": <number-of-dimensions>,
"similarity": "euclidean | cosine | dotProduct",
"quantization": "none | scalar | binary", // Optional
"hnswOptions": { // Optional (Preview feature)
"maxEdges": <number-of-connected-neighbors>,
"numEdgeCandidates": <number-of-nearest-neighbors>
}
},
{
"type": "filter", // Optional: for pre-filtering
"path": "<field-to-index>"
}
]
}
```
**Note**: The exact syntax for creating indexes varies by driver/interface. The above shows the core index definition structure that applies across all methods.
### Basic Definition
Most vector search indexes only need the vector field:
```javascript
{
"fields": [
{
"type": "vector",
"path": "<embedding-field>",
"numDimensions": <number>,
"similarity": "<similarity-function>"
}
]
}
```
---
## Index Configuration Parameters
### Required: numDimensions
**Definition**: Number of dimensions in your vector embeddings. MongoDB enforces this at both index-time and query-time.
**Constraints**:
- Must be less than or equal to 8192
- For int1 (binary) vectors: MUST be a multiple of 8
- For int8 vectors: 1 to 8192
- For float32 vectors: 1 to 8192
**How to Determine**:
- The embedding model determines this value
- It MUST match the actual dimension count of your vectors
- Cannot be changed after index creation (requires dropping and recreating index)
**Example - Voyage AI Models**:
- voyage-3-large: 2048 dimensions
- voyage-4: Configurable output dimensions (256, 512, 1024, 2048, 4096)
---
### Required: similarity
**Definition**: The similarity function used to compare vectors and rank results.
**Available Options**:
| Similarity | Score Formula | Score Range | Best For | Requirements |
|-----------|---------------|-------------|----------|--------------|
| `cosine` | `(1 + cosine(v1,v2)) / 2` | [0, 1] | Most embedding models, normalized vectors | Cannot use zero-magnitude vectors |
| `dotProduct` | `(1 + dotProduct(v1,v2)) / 2` | [0, 1] | **Most efficient** - angle + magnitude | Vectors MUST be normalized to unit length |
| `euclidean` | `1 / (1 + euclidean(v1,v2))` | [0, 1] | Spatial/geometric similarity | **REQUIRED** for int1 (binary) quantized vectors |
**Decision Process**:
1. Check your embedding model documentation for recommended similarity function
2. If model produces normalized vectors -> use `dotProduct` (fastest)
3. If model does NOT normalize vectors -> use `cosine`
4. If using binary quantization (int1) -> MUST use `euclidean`
5. When uncertain -> start with `dotProduct` and normalize your vectors
**Notes**:
- All functions return scores in range [0, 1] where 1 = most similar
- `dotProduct` is most efficient but requires normalized vectors
- Check embedding model documentation for recommendations
---
### Optional: quantization
**Definition**: Automatic vector compression to reduce storage and improve query speed at the cost of some accuracy.
**Syntax**:
```javascript
{
"type": "vector",
"path": "<field>",
"numDimensions": <number>,
"similarity": "<function>",
"quantization": "none | scalar | binary"
}
```
**Options**:
| Type | Compression | Accuracy | Storage | Use Case |
|------|-------------|----------|---------|----------|
| `none` | 1x (no compression) | Highest | Full size | Maximum accuracy needed, small datasets (less than 1M vectors) |
| `scalar` | 4x | High | 4x smaller | Good balance for most cases (1M-10M+ vectors) |
| `binary` | 4-8x | Good | Maximum compression | Large datasets (10M+ vectors), speed priority |
**Important Rules**:
- `none`: Default if omitted. Use for pre-quantized vectors (int1, int8)
- `scalar`: Transforms float32/double values to 1-byte integers
- `binary`: Transforms values to single bit. numDimensions MUST be multiple of 8
- Binary quantization REQUIRES `euclidean` similarity
- Only use with float32 or double vectors (NOT with pre-quantized int1/int8)
**Example**:
```javascript
{
"type": "vector",
"path": "plot_embedding",
"numDimensions": 1536,
"similarity": "cosine",
"quantization": {
"type": "scalar"
}
}
```
---
### Optional: hnswOptions (Preview Feature)
**Definition**: Parameters for the Hierarchical Navigable Small Worlds graph construction algorithm.
**Warning**: Modifying default values might negatively impact your index and queries. Use with caution.
**Syntax**:
```javascript
{
"type": "vector",
"path": "<field>",
"numDimensions": <number>,
"similarity": "<function>",
"hnswOptions": {
"maxEdges": <16-64>, // Default: 16
"numEdgeCandidates": <100-3200> // Default: 100
}
}
```
**Parameters**:
**maxEdges** (16-64, default: 16):
- Maximum number of connections per node in the graph
- Higher values:
- Better recall (finds more relevant results)
- Slower queries (more neighbors to evaluate)
- More memory usage (more connections stored)
- Slower indexing (more neighbors to adjust)
**numEdgeCandidates** (100-3200, default: 100):
- Maximum nodes evaluated to find best connections for new nodes
- Higher values:
- Better graph quality (improves search accuracy)
- Can negatively affect query latency
**Recommendation**: Leave at defaults unless you have specific performance requirements and understand the trade-offs.
---
## Filter Fields (Pre-filtering)
### About Filter Fields
**Definition**: Additional fields indexed to enable pre-filtering before vector similarity computation. This narrows the search scope and improves performance.
**Use Case**: Filter by specific criteria (e.g., category, date range, user ID) BEFORE computing vector similarity.
**Performance**: Filtering before similarity computation is much faster than post-filtering with `$match`.
**Supported Field Types**: boolean, date, objectId, numeric (int32, int64, double), string, UUID, and arrays of these types.
---
### Syntax
```javascript
{
"fields": [
{
"type": "vector",
"path": "embedding",
"numDimensions": 1024,
"similarity": "cosine"
},
{
"type": "filter",
"path": "category" // String field for filtering
},
{
"type": "filter",
"path": "year" // Numeric field for filtering
}
]
}
```
---
### When to Use Filter Fields
**Use filter fields when**:
- You need to filter by exact values (category = "Action")
- You need range filtering (year >= 2020)
- Filter criteria are known at query time
- You want maximum query performance (filters before computing similarity)
- You have multi-tenant data that needs isolation
**Use post-filtering ($match) when**:
- Filters are ad-hoc and change frequently
- Complex aggregation logic is needed
- Fields are not worth indexing (rarely used)
- Combining with other aggregation stages
---
### Supported Filter Operators
MongoDB Vector Search supports the following MQL operators in the `filter` option:
| Type | Operators |
|------|-----------|
| Equality | `$eq`, `$ne` |
| Range | `$gt`, `$lt`, `$gte`, `$lte` |
| In set | `$in`, `$nin` |
| Existence | `$exists` |
| Logical | `$not`, `$nor`, `$and`, `$or` |
**Note**: Other query operators, aggregation pipeline operators, and MongoDB Search operators are NOT supported in the filter option.
---
### Filter Examples
**Index with filter fields**:
```javascript
{
"fields": [
{
"type": "vector",
"path": "plot_embedding",
"numDimensions": 2048,
"similarity": "dotProduct"
},
{
"type": "filter",
"path": "genres" // String or array of strings
},
{
"type": "filter",
"path": "year" // Numeric field
}
]
}
```
**Query with single filter**:
```javascript
{
$vectorSearch: {
queryVector: [<array-of-numbers>],
path: "plot_embedding",
filter: {
genres: { $eq: "Action" }
},
numCandidates: 150,
limit: 10
}
}
```
**Query with multiple filters using $and**:
```javascript
{
$vectorSearch: {
queryVector: [<array-of-numbers>],
path: "plot_embedding",
filter: {
$and: [
{ genres: "Action" },
{ year: { $gte: 2020 } }
]
},
numCandidates: 150,
limit: 10
}
}
```
**Short form of $eq** (recommended):
```javascript
{
$vectorSearch: {
queryVector: [<array-of-numbers>],
path: "plot_embedding",
filter: {
genres: "Action", // Equivalent to { genres: { $eq: "Action" } }
year: { $gte: 2020 }
},
numCandidates: 150,
limit: 10
}
}
```
---
### Important Notes
**Pre-filtering does NOT affect scores**: The vectorSearchScore returned for documents is based only on vector similarity, not on how well they matched the filter criteria.
**Filter fields must be indexed**: You must add fields as type "filter" in your index definition to use them in the filter option. Fields not indexed cannot be used for pre-filtering.
**Arrays are supported**: You can filter on fields that contain arrays. MongoDB automatically handles array matching.
---
## Query Construction
### $vectorSearch Stage
**Definition**: The `$vectorSearch` stage performs semantic search for a query vector on indexed vector fields. It must be the first stage in an aggregation pipeline.
**Requirements**:
- Atlas cluster running MongoDB v6.0.11, v7.0.2, or later
- A vector search index on the collection with vector-type fields
- `$vectorSearch` MUST be the first stage in the pipeline
---
### Basic Query Syntax
```javascript
{
"$vectorSearch": {
"index": "<index-name>",
"path": "<field-to-search>",
"queryVector": [<array-of-numbers>],
"numCandidates": <number-of-candidates>,
"limit": <number-of-results>,
"filter": {<filter-specification>}, // Optional
"exact": true | false // Optional
}
}
```
---
### Required Fields
**index** (String, Required):
- Name of the MongoDB Vector Search index to use
- MongoDB returns no results if the index name is misspelled or doesn't exist
- Must match the name specified when creating the index
**path** (String, Required):
- Name of the indexed vector field to search
- Must be a field indexed as type "vector" in your index definition
- Use dot notation for nested fields (e.g., "metadata.embedding")
**queryVector** (Array of Numbers, Required):
- Array of numbers representing your query vector
- Can be float32, BSON BinData float32, or BSON BinData int1/int8
- Array size MUST match numDimensions specified in the index
- You must use the same embedding model that generated the indexed vectors
**limit** (Integer, Required):
- Number of documents to return in results
- Must be an integer value
- Cannot exceed numCandidates if numCandidates is specified
---
### Conditional Fields
**numCandidates** (Integer, Conditional):
- Number of nearest neighbors to use during ANN search
- Required if `exact` is false or omitted
- Must be less than or equal to 10000
- Cannot be less than `limit`
- Recommended: Set to at least 20x the `limit` value for good recall
**Example**:
```javascript
{
$vectorSearch: {
queryVector: [<array>],
path: "embedding",
numCandidates: 150, // 15x the limit
limit: 10
}
}
```
---
### Optional Fields
**filter** (Object, Optional):
- MQL expression to pre-filter documents before vector search
- Only works with fields indexed as type "filter"
- Supported operators: $eq, $ne, $gt, $lt, $gte, $lte, $in, $nin, $exists, $and, $or, $not, $nor
- See Filter Fields section for details and examples
**exact** (Boolean, Optional):
- Set to `true` for ENN (Exact Nearest Neighbor) search
- Set to `false` or omit for ANN (Approximate Nearest Neighbor) search
- Default: false
**ENN vs ANN**:
- **ANN (default)**: Faster, uses HNSW algorithm, good for large datasets, 90-95% recall
- **ENN**: Exhaustive search, guaranteed exact matches, slower, use for small datasets (less than 10K docs) or measuring accuracy baseline
---
### Complete Query Examples
**Basic ANN query**:
```javascript
db.collection.aggregate([
{
$vectorSearch: {
index: "vector_index",
path: "plot_embedding",
queryVector: [<1536-dimension-array>],
numCandidates: 150,
limit: 10
}
},
{
$project: {
_id: 0,
title: 1,
plot: 1,
score: { $meta: "vectorSearchScore" }
}
}
])
```
**ANN query with pre-filtering**:
```javascript
db.collection.aggregate([
{
$vectorSearch: {
index: "vector_index",
path: "plot_embedding",
queryVector: [<2048-dimension-array>],
filter: {
$and: [
{ year: { $gte: 1955 } },
{ year: { $lt: 1975 } }
]
},
numCandidates: 150,
limit: 10
}
},
{
$project: {
_id: 0,
title: 1,
year: 1,
score: { $meta: "vectorSearchScore" }
}
}
])
```
**ENN query (exact search)**:
```javascript
db.collection.aggregate([
{
$vectorSearch: {
index: "vector_index",
path: "plot_embedding",
queryVector: [<2048-dimension-array>],
exact: true,
limit: 10
}
},
{
$project: {
_id: 0,
title: 1,
score: { $meta: "vectorSearchScore" }
}
}
])
```
---
### Retrieving Vector Search Scores
Use `$meta: "vectorSearchScore"` in a `$project` stage to include similarity scores:
```javascript
{
$project: {
title: 1,
score: { $meta: "vectorSearchScore" }
}
}
```
**Important**:
- Scores are in range [0, 1] where 1 = most similar
- You can ONLY use `vectorSearchScore` after a `$vectorSearch` stage
- Pre-filtering does NOT affect the score (only vector similarity affects score)
---
### Post-filtering with $match
For ad-hoc filters or complex logic not indexed as filter fields, use `$match` after `$vectorSearch`:
```javascript
db.collection.aggregate([
{
$vectorSearch: {
index: "vector_index",
path: "plot_embedding",
queryVector: [<array>],
numCandidates: 150,
limit: 50 // Get more candidates for post-filtering
}
},
{
$match: {
category: "Electronics",
"reviews.rating": { $gte: 4.5 } // Complex nested field
}
},
{ $limit: 10 }
])
```
**Performance Note**: Post-filtering is slower than pre-filtering because it computes similarity for all candidates first.
---
## Query Optimization
### numCandidates Tuning
**Definition**: The `numCandidates` parameter controls the trade-off between recall (finding relevant results) and query performance in ANN searches.
**Rule of Thumb**: A good starting point is 20x your `limit` value. You can adjust between 10-20x (or higher) based on your recall and performance requirements.
**Example**:
```javascript
{
$vectorSearch: {
queryVector: [<array>],
path: "embedding",
numCandidates: 200, // 20x the limit — good starting point; tune between 10-50x based on recall and latency requirements
limit: 10
}
}
```
---
### When to Adjust numCandidates
**Increase when**:
- Search results miss relevant documents
- Large dataset (millions of vectors)
- Using quantized vectors (int8 or int1)
- Heavy pre-filtering is applied
**Decrease when**:
- Queries are too slow and results are already good
- Small dataset (thousands of vectors)
- Speed is more important than perfect recall
**Note on low limit values**: A very low limit (e.g., 5) may need proportionally higher numCandidates (e.g., 40x) to maintain recall.
### Test and Measure
- Start with 20x limit and run sample queries
- Check result quality and query latency
- Adjust up or down based on your accuracy vs performance requirements
---
### ANN vs ENN Search
**ANN (Approximate Nearest Neighbor)**:
- Default search method
- Uses HNSW algorithm for fast approximate search
- Typically 90-95% recall (finds 90-95% of exact matches)
- Much faster than ENN for large datasets
- Requires `numCandidates` parameter
**Use ANN when**:
- You have production queries
- Dataset is large (more than 10K documents)
- 90-95% recall is acceptable
- Query speed is important
**ENN (Exact Nearest Neighbor)**:
- Exhaustive search of all indexed vectors
- Guaranteed to find exact best matches
- Much slower than ANN
- Set `exact: true` in query
- Does NOT require `numCandidates` parameter
- Uses full-fidelity vectors even when quantization is enabled
**Use ENN when**:
- Measuring accuracy baseline (ground truth for testing)
- Collection has less than 10K documents
- Very selective filters (less than 5% of data matches)
- You need guaranteed best matches
---
### Pre-filtering vs Post-filtering Performance
**Pre-filtering (filter option)**:
- Fastest: Filters BEFORE computing similarity
- Use for exact matches, range queries, known criteria
- Requires fields indexed as type "filter"
- Limited to supported MQL operators ($eq, $ne, $gt, $lt, $gte, $lte, $in, $nin, $exists, $and, $or, $not, $nor)
**Post-filtering ($match stage)**:
- Slower: Computes similarity for all candidates first
- Use for ad-hoc filters, complex logic, unindexed fields
- Full MQL operator support
- Can combine with other aggregation stages
**Recommendation**: Use pre-filtering whenever possible for best performance. Reserve post-filtering for complex or ad-hoc queries.
---
### Parallel Query Execution
MongoDB Vector Search parallelizes query execution across segments when running on dedicated search nodes, which can improve response time for queries on large datasets.
**Notes**:
- Works automatically on dedicated search nodes
- High-CPU systems provide more performance improvement
- Not guaranteed for every query (e.g., when too many concurrent queries are queued)
- May cause slight inconsistencies in results for successive identical queries
**If you see inconsistent results**: Increase `numCandidates` to improve consistency.
---
### Best Practices Summary
1. **Start with numCandidates = 20x limit**: Provides good balance of recall and performance
2. **Use pre-filtering when possible**: Index filter fields for known filtering criteria
3. **Choose appropriate similarity function**: Match your embedding model's recommendations
4. **Consider quantization for large datasets**: Use scalar or binary quantization for 10M+ vectors
5. **Use ANN for production**: Reserve ENN for testing/small datasets
6. **Test with your data**: Run sample queries and measure recall vs latency
7. **Monitor and adjust**: Use query performance metrics to tune numCandidates
8. **Match query vectors to index**: Use the same embedding model and dimensions
---
## Vector Search on Views
Version requirements, supported stages, limitations, and troubleshooting are identical to Atlas Search on Views — see `lexical-search-indexing.md`. The difference is using a `vectorSearch`-type index and querying with `$vectorSearch`.
**Example: partial index (exclude documents without embeddings)**
```javascript
db.createView("moviesWithEmbeddings", "embedded_movies", [
{
$match: {
$expr: { $ne: [{ $type: "$plot_embedding_voyage_3_large" }, "missing"] }
}
}
])
db.moviesWithEmbeddings.createSearchIndex(
"embeddingsIndex",
"vectorSearch",
{
"fields": [
{
"type": "vector",
"numDimensions": 2048,
"path": "plot_embedding_voyage_3_large",
"similarity": "cosine"
}
]
}
)
// 8.1+: query view directly; 8.0: query source collection using index name
db.moviesWithEmbeddings.aggregate([
{
$vectorSearch: {
index: "embeddingsIndex",
path: "plot_embedding_voyage_3_large",
queryVector: [<query-vector-2048-dimensions>],
numCandidates: 100,
limit: 10
}
}
])
```
---
SKILL.md
---
name: mongodb-search-and-ai
description: |
Guides MongoDB users through implementing and optimizing Atlas Search (full-text), Vector Search (semantic), and Hybrid Search solutions. Use this skill when users need to build search functionality for text-based queries (autocomplete, fuzzy matching, faceted search), semantic similarity (embeddings, RAG applications), or combined approaches. Also use when users need text containment, substring matching ('contains', 'includes', 'appears in'), case-insensitive or multi-field text search, or filtering across many fields with variable combinations. Provides workflows for selecting the right search type, creating indexes, constructing queries, and optimizing performance using the MongoDB MCP server.
license: Apache-2.0
metadata:
version: "1.0.0"
---
# MongoDB Search and AI Recommendations Skill
You are helping MongoDB users implement, optimize, and troubleshoot Atlas Search (lexical), Vector Search (semantic), and Hybrid Search (combined) solutions. Your goal is to understand their use case, recommend the appropriate search approach, and help them build effective indexes and queries.
## Core Principles
1. **Understand before building** - Validate the use case to ensure you recommend the right solution
2. **Always inspect first** - Check existing indexes and schema before making recommendations
3. **Explain before executing** - Describe what indexes will be created and require explicit approval
4. **Optimize for the use case** - Different use cases require different index configurations and query patterns
5. **Handle read-only scenarios** - If you do not have access to `create`, `update`, or `delete` operation tools, you are in read-only mode. Provide the complete index configuration JSON so the user can create it themselves, including via the Atlas UI.
## Workflow
### 1. Discovery Phase
**Check the environment:**
- Use `list-databases` and `list-collections` to understand available data
- If the user mentions a collection, use `collection-schema` to inspect field structure
- Use `collection-indexes` to see existing indexes
- Use `atlas-inspect-cluster` to determine the cluster's MongoDB version
**Understand the use case:**
If the user's request is vague:
- Ask clarifying questions about their needs
- Infer likely collection and fields from schema
- Confirm understanding before proceeding
Common questions to ask:
- What are users searching for? (products, movies, documents, etc.)
- What fields contain the searchable content?
- Do they need exact matching, fuzzy matching, or semantic similarity?
- Do they need filters (price ranges, categories, dates)?
- Do they need autocomplete/typeahead functionality?
### 2. Determine Search Type
**Atlas Search (Lexical/Full-Text):**
Use when users need:
- Keyword matching with relevance scoring
- Fuzzy matching for typo tolerance
- Autocomplete/typeahead
- Faceted search with filters
- Language-specific text analysis
- Token-based search
- Lexical search with views
**Vector Search (Semantic):**
Use when users need:
- Semantic similarity ("find movies about coming of age stories")
- Natural language understanding
- RAG (Retrieval Augmented Generation) applications
- Finding conceptually similar items
- Cross-modal search
- Vector search with views
**Hybrid Search:**
Use when users need:
- Combining multiple search approaches (e.g., vector + lexical, multiple text searches)
- Queries like "find action movies similar to 'epic space battles'" (combining keyword filtering with semantic similarity)
- Results that factor in multiple relevance criteria
- Uses `$rankFusion` (rank-based) or `$scoreFusion` (score-based) to merge pipelines
### 3. Version Check (Hybrid Search only)
If the search type is **Hybrid using `$rankFusion` or `$scoreFusion`**, verify the cluster version before proceeding:
- `$rankFusion` requires MongoDB 8.0+
- `$scoreFusion` requires MongoDB 8.2+
If the version requirement is not met, do not proceed — inform the user the feature is unavailable and suggest upgrading. Do not consult `references/hybrid-search.md`.
If the search type is Lexical, Vector, or the lexical prefilter pattern (`vectorSearch` operator inside `$search`), proceed to the next step.
### 4. Consult Reference Files
Always consult the appropriate reference file(s) before recommending indexes or queries:
- **Lexical**: consult both `references/lexical-search-indexing.md` (index) and `references/lexical-search-querying.md` (query)
- **Vector**: consult `references/vector-search.md`
- **Hybrid**: consult `references/hybrid-search.md` (and the lexical/vector files for the individual pipeline stages within it)
### 5. Execution and Validation
**Creating indexes:**
1. Explain the index configuration in plain language
2. Show the JSON structure
3. Ask what the user wants to name the index
4. Get explicit approval: "Should I create this index?"
5. Use MCP's `create-index` tool after approval
6. In read-only mode, provide the complete index JSON for creation via the Atlas UI
**Running queries:**
1. Show the aggregation pipeline
2. Execute using MCP's `aggregate` tool
3. Present results clearly
**Refining existing queries:**
1. Ask the user to share their current query
2. Compare against the query patterns and best practices in the relevant reference file(s)
3. Propose specific improvements with before/after examples
4. Run the revised query with `aggregate` to validate the results
## Anti-Patterns to Avoid
**NEVER recommend $regex or $text for search use cases:**
- **$regex**: Not designed for full-text search. Lacks relevance scoring, fuzzy matching, and language-aware tokenization.
- **$text**: Legacy operator that doesn't scale well for search workloads.
If a user asks for regex/text for a search use case, explain why Atlas Search is more appropriate and show the equivalent pattern.
## Handling Edge Cases
**User mentions fields you can't find:**
- Use `collection-schema` to inspect available fields
- Suggest alternatives or ask for clarification
**Required field doesn't exist:**
- Explain what needs to be added and how (e.g., embedding field for vector search)
**Query fails or index missing:**
- Use `collection-indexes` to verify index exists
- If missing, explain index needs to be created first
**Multiple collections are relevant:**
- List options and ask which one they mean
- If context makes it obvious, confirm your assumption
## Remember
- Always check existing indexes before recommending new ones
- Explain technical concepts in accessible language
- Require approval before creating indexes
- Map user's business requirements to technical implementations
- Use the appropriate search type for the use case