You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* [DOCS] Update ESQL metadata fields page
**esql-metadata-fields.md:**
- restructured from bullet list to table format for metadata fields
- added `_index_mode` and `_source` fields to available metadata
- improved field descriptions (more detailed)
- added "usage and limitations" section
- reorganized examples into subsections with headers
- added `_score` sorting example
- added tip box linking to search documentation
* 🚙Drive by updates to search functions ref page
moved tutorial link into tip box at top
added cross-reference to search overview documentation
minor text flow improvements and punctuation fixes
* Fix id typo
* Apply suggestions from review
Co-authored-by: Bogdan Pintea <[email protected]>
Copy file name to clipboardExpand all lines: docs/reference/query-languages/esql/esql-metadata-fields.md
+41-11Lines changed: 41 additions & 11 deletions
Original file line number
Diff line number
Diff line change
@@ -6,24 +6,40 @@ mapped_pages:
6
6
7
7
# {{esql}} metadata fields [esql-metadata-fields]
8
8
9
+
{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md).
9
10
10
-
{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md). The currently supported ones are:
11
-
12
-
*[`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md): the index to which the document belongs. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
13
-
*[`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md): the source document’s ID. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
14
-
*`_version`: the source document’s version. The field is of the type [long](/reference/elasticsearch/mapping-reference/number.md).
15
-
*[`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md): the ignored source document fields. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
16
-
*`_score`: when enabled, the final score assigned to each row matching an ES|QL query. Scoring will be updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md).
17
-
18
-
To enable the access to these fields, the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command needs to be provided with a dedicated directive:
11
+
To access these fields, use the `METADATA` directive with the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command. For example:
19
12
20
13
```esql
21
14
FROM index METADATA _index, _id
22
15
```
23
16
24
-
Metadata fields are only available if the source of the data is an index. Consequently, `FROM` is the only source commands that supports the `METADATA` directive.
17
+
## Available metadata fields
18
+
19
+
The following metadata fields are available in {{esql}}:
|[`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md)|[keyword](/reference/elasticsearch/mapping-reference/keyword.md)| Names every field in a document that was ignored when the document was indexed. |
25
+
|[`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md)|[keyword](/reference/elasticsearch/mapping-reference/keyword.md)| Index name. |
26
+
|`_index_mode`|[keyword](/reference/elasticsearch/mapping-reference/keyword.md)|[Index mode](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting). For example: `standard`, `lookup`, or `logsdb`. |
27
+
|`_score`|[`float`](/reference/elasticsearch/mapping-reference/number.md)| Query relevance score (when enabled). Scores are updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md). |
28
+
|[`_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md)| Special `_source` type | Original JSON document body passed at index time (or a reconstructed version if [synthetic `_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) is enabled). |
29
+
|`_version`|[`long`](/reference/elasticsearch/mapping-reference/number.md)| Document version number |
30
+
31
+
## Usage and limitations
32
+
33
+
- Metadata fields are only available when the data source is an index
34
+
- The `_source` type is not supported by functions
35
+
- Only the `FROM` command supports the `METADATA` directive
36
+
- Once enabled, metadata fields work like regular index fields
37
+
38
+
## Examples
25
39
26
-
Once enabled, these fields will be available to subsequent processing commands, just like other index fields:
40
+
### Basic metadata usage
41
+
42
+
Once enabled, metadata fields are available to subsequent processing commands, just like other index fields:
27
43
28
44
```esql
29
45
FROM ul_logs, apps METADATA _index, _version
@@ -40,6 +56,8 @@ FROM ul_logs, apps METADATA _index, _version
40
56
| 14 | apps | 1 | apps_14 |
41
57
| 14 | ul_logs | 1 | ul_logs_14 |
42
58
59
+
### Metadata fields and aggregations
60
+
43
61
Similar to index fields, once an aggregation is performed, a metadata field will no longer be accessible to subsequent commands, unless used as a grouping field:
44
62
45
63
```esql
@@ -51,3 +69,15 @@ FROM employees METADATA _index, _id
51
69
| --- | --- |
52
70
| 10100 | employees |
53
71
72
+
### Sort results by search score
73
+
74
+
```esql
75
+
FROM products METADATA _score
76
+
| WHERE MATCH(description, "wireless headphones")
77
+
| SORT _score DESC
78
+
| KEEP name, description, _score
79
+
```
80
+
81
+
:::{tip}
82
+
Refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring) for more information on relevance scoring and how to use `_score` in your queries.
For a high-level overview of search functionalities in {{esql}}, and to learn about relevance scoring, refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring).
14
+
:::
15
+
16
+
{{esql}} provides a set of functions for performing searching on text fields.
17
+
9
18
Use these functions
10
19
for [full-text search](docs-content://solutions/search/full-text.md)
11
20
and [semantic search](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md).
12
21
13
-
Get started with {{esql}} for search use cases with
0 commit comments