Skip to content

Commit 4df0e99

Browse files
leemthompobpintea
andauthored
[DOCS] Update ESQL metadata fields page (#129939)
* [DOCS] Update ESQL metadata fields page **esql-metadata-fields.md:** - restructured from bullet list to table format for metadata fields - added `_index_mode` and `_source` fields to available metadata - improved field descriptions (more detailed) - added "usage and limitations" section - reorganized examples into subsections with headers - added `_score` sorting example - added tip box linking to search documentation * 🚙Drive by updates to search functions ref page moved tutorial link into tip box at top added cross-reference to search overview documentation minor text flow improvements and punctuation fixes * Fix id typo * Apply suggestions from review Co-authored-by: Bogdan Pintea <[email protected]>
1 parent 138f350 commit 4df0e99

File tree

2 files changed

+51
-15
lines changed

2 files changed

+51
-15
lines changed

docs/reference/query-languages/esql/esql-metadata-fields.md

Lines changed: 41 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -6,24 +6,40 @@ mapped_pages:
66

77
# {{esql}} metadata fields [esql-metadata-fields]
88

9+
{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md).
910

10-
{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md). The currently supported ones are:
11-
12-
* [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md): the index to which the document belongs. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
13-
* [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md): the source document’s ID. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
14-
* `_version`: the source document’s version. The field is of the type [long](/reference/elasticsearch/mapping-reference/number.md).
15-
* [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md): the ignored source document fields. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
16-
* `_score`: when enabled, the final score assigned to each row matching an ES|QL query. Scoring will be updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md).
17-
18-
To enable the access to these fields, the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command needs to be provided with a dedicated directive:
11+
To access these fields, use the `METADATA` directive with the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command. For example:
1912

2013
```esql
2114
FROM index METADATA _index, _id
2215
```
2316

24-
Metadata fields are only available if the source of the data is an index. Consequently, `FROM` is the only source commands that supports the `METADATA` directive.
17+
## Available metadata fields
18+
19+
The following metadata fields are available in {{esql}}:
20+
21+
| Metadata field | Type | Description |
22+
|---------------|------|-------------|
23+
| [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Unique document ID. |
24+
| [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Names every field in a document that was ignored when the document was indexed. |
25+
| [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Index name. |
26+
| `_index_mode` | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | [Index mode](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting). For example: `standard`, `lookup`, or `logsdb`. |
27+
| `_score` | [`float`](/reference/elasticsearch/mapping-reference/number.md) | Query relevance score (when enabled). Scores are updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md). |
28+
| [`_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md) | Special `_source` type | Original JSON document body passed at index time (or a reconstructed version if [synthetic `_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) is enabled). |
29+
| `_version` | [`long`](/reference/elasticsearch/mapping-reference/number.md) | Document version number |
30+
31+
## Usage and limitations
32+
33+
- Metadata fields are only available when the data source is an index
34+
- The `_source` type is not supported by functions
35+
- Only the `FROM` command supports the `METADATA` directive
36+
- Once enabled, metadata fields work like regular index fields
37+
38+
## Examples
2539

26-
Once enabled, these fields will be available to subsequent processing commands, just like other index fields:
40+
### Basic metadata usage
41+
42+
Once enabled, metadata fields are available to subsequent processing commands, just like other index fields:
2743

2844
```esql
2945
FROM ul_logs, apps METADATA _index, _version
@@ -40,6 +56,8 @@ FROM ul_logs, apps METADATA _index, _version
4056
| 14 | apps | 1 | apps_14 |
4157
| 14 | ul_logs | 1 | ul_logs_14 |
4258

59+
### Metadata fields and aggregations
60+
4361
Similar to index fields, once an aggregation is performed, a metadata field will no longer be accessible to subsequent commands, unless used as a grouping field:
4462

4563
```esql
@@ -51,3 +69,15 @@ FROM employees METADATA _index, _id
5169
| --- | --- |
5270
| 10100 | employees |
5371

72+
### Sort results by search score
73+
74+
```esql
75+
FROM products METADATA _score
76+
| WHERE MATCH(description, "wireless headphones")
77+
| SORT _score DESC
78+
| KEEP name, description, _score
79+
```
80+
81+
:::{tip}
82+
Refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring) for more information on relevance scoring and how to use `_score` in your queries.
83+
:::

docs/reference/query-languages/esql/functions-operators/search-functions.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,21 +6,27 @@ mapped_pages:
66

77
# {{esql}} Search functions [esql-search-functions]
88

9+
:::{tip}
10+
Get started with {{esql}} for search use cases with
11+
our [hands-on tutorial](docs-content://solutions/search/esql-search-tutorial.md).
12+
13+
For a high-level overview of search functionalities in {{esql}}, and to learn about relevance scoring, refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring).
14+
:::
15+
16+
{{esql}} provides a set of functions for performing searching on text fields.
17+
918
Use these functions
1019
for [full-text search](docs-content://solutions/search/full-text.md)
1120
and [semantic search](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md).
1221

13-
Get started with {{esql}} for search use cases with
14-
our [hands-on tutorial](docs-content://solutions/search/esql-search-tutorial.md).
15-
1622
Full text functions can be used to
1723
match [multivalued fields](/reference/query-languages/esql/esql-multivalued-fields.md).
1824
A multivalued field that contains a value that matches a full text query is
1925
considered to match the query.
2026

2127
Full text functions are significantly more performant for text search use cases
2228
on large data sets than using pattern matching or regular expressions with
23-
`LIKE` or `RLIKE`
29+
`LIKE` or `RLIKE`.
2430

2531
See [full text search limitations](/reference/query-languages/esql/limitations.md#esql-limitations-full-text-search)
2632
for information on the limitations of full text search.

0 commit comments

Comments
 (0)