Skip to content

Commit 94aaa1f

Browse files
kosabogiszabosteveyetanothertw
authored
Adds Rollup API examples to 'Get started with rollups using the API' page (#1893)
### 📸 [Preview](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/1893/manage-data/lifecycle/rollup/getting-started-api#historical-only-search-example) This PR adds the Rollup search API examples to the Get started with rollups using the API page, as part of the approach to unify API reference documentation. Related issue: elastic/docs-projects#302 --------- Co-authored-by: István Zoltán Szabó <[email protected]> Co-authored-by: Vlada Chirmicci <[email protected]>
1 parent 0f5ff15 commit 94aaa1f

File tree

1 file changed

+173
-2
lines changed

1 file changed

+173
-2
lines changed

manage-data/lifecycle/rollup/getting-started-api.md

Lines changed: 173 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -272,8 +272,179 @@ Which returns a corresponding response:
272272

273273
In addition to being more complicated (date histogram and a terms aggregation, plus an additional average metric), you’ll notice the date_histogram uses a `7d` interval instead of `60m`.
274274

275+
This quickstart should have provided a concise overview of the core functionality that Rollup exposes. There are more tips and things to consider when setting up Rollups, which you can find throughout the rest of this section. You may also explore the [REST API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-rollup-get-jobs) for an overview of what is available.
275276

276-
## Conclusion [_conclusion]
277+
## Historical-only search example
277278

278-
This quickstart should have provided a concise overview of the core functionality that Rollup exposes. There are more tips and things to consider when setting up Rollups, which you can find throughout the rest of this section. You may also explore the [REST API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-rollup-get-jobs) for an overview of what is available.
279+
Suppose you have an index named `sensor-1` that contains raw data, and you've created a {{rollup-job}} with the following configuration:
280+
281+
```console
282+
PUT _rollup/job/sensor
283+
{
284+
"index_pattern": "sensor-*",
285+
"rollup_index": "sensor_rollup",
286+
"cron": "*/30 * * * * ?",
287+
"page_size": 1000,
288+
"groups": {
289+
"date_histogram": {
290+
"field": "timestamp",
291+
"fixed_interval": "1h",
292+
"delay": "7d"
293+
},
294+
"terms": {
295+
"fields": [ "node" ]
296+
}
297+
},
298+
"metrics": [
299+
{
300+
"field": "temperature",
301+
"metrics": [ "min", "max", "sum" ]
302+
},
303+
{
304+
"field": "voltage",
305+
"metrics": [ "avg" ]
306+
}
307+
]
308+
}
309+
```
310+
% TEST[setup:sensor_index]
311+
312+
This rolls up the `sensor-*` pattern and stores the results in `sensor_rollup`.
313+
To search this rolled up data, use the `_rollup_search` endpoint.
314+
You can use Query DSL to search the rolled-up data:
315+
316+
```console
317+
GET /sensor_rollup/_rollup_search
318+
{
319+
"size": 0,
320+
"aggregations": {
321+
"max_temperature": {
322+
"max": {
323+
"field": "temperature"
324+
}
325+
}
326+
}
327+
}
328+
```
329+
% TEST[setup:sensor_prefab_data]
330+
% TEST[s/_rollup_search/_rollup_search?filter_path=took,timed_out,terminated_early,_shards,hits,aggregations/]
331+
332+
The query is targeting the `sensor_rollup` data, since this contains the rollup
333+
data as configured in the job. A `max` aggregation has been used on the
334+
`temperature` field, yielding the following response:
335+
336+
```console-result
337+
GET /sensor_rollup/_rollup_search
338+
{
339+
"size": 0,
340+
"aggregations": {
341+
"max_temperature": {
342+
"max": {
343+
"field": "temperature"
344+
}
345+
}
346+
}
347+
}
348+
```
349+
% TESTRESPONSE[s/"took" : 102/"took" : $body.$_path/]
350+
% TESTRESPONSE[s/"_shards" : \.\.\. /"_shards" : $body.$_path/]
279351

352+
The response follows the same structure as a standard query with aggregations: it includes metadata about the request (`took`, `_shards`, etc.), an empty hits section (as rollup searches do not return individual documents), and the aggregation results.
353+
354+
Rollup searches are limited to the functionality defined in the {{rollup-job}} configuration. For example, if the `avg` metric was not configured for the `temperature` field, calculating the average temperature is not possible. Running such a query results in an error:
355+
356+
```console
357+
GET sensor_rollup/_rollup_search
358+
{
359+
"size": 0,
360+
"aggregations": {
361+
"avg_temperature": {
362+
"avg": {
363+
"field": "temperature"
364+
}
365+
}
366+
}
367+
}
368+
```
369+
% TEST[continued]
370+
% TEST[catch:/illegal_argument_exception/]
371+
372+
```console-result
373+
{
374+
"error": {
375+
"root_cause": [
376+
{
377+
"type": "illegal_argument_exception",
378+
"reason": "There is not a rollup job that has a [avg] agg with name [avg_temperature] which also satisfies all requirements of query.",
379+
"stack_trace": ...
380+
}
381+
],
382+
"type": "illegal_argument_exception",
383+
"reason": "There is not a rollup job that has a [avg] agg with name [avg_temperature] which also satisfies all requirements of query.",
384+
"stack_trace": ...
385+
},
386+
"status": 400
387+
}
388+
```
389+
% TESTRESPONSE[s/"stack_trace": \.\.\./"stack_trace": $body.$_path/]
390+
391+
## Searching both historical rollup and non-rollup data
392+
393+
The rollup search API has the capability to search across both live
394+
non-rollup data and the aggregated rollup data. This is done by adding
395+
the live indices to the URI:
396+
397+
```console
398+
GET sensor-1,sensor_rollup/_rollup_search
399+
{
400+
"size": 0,
401+
"aggregations": {
402+
"max_temperature": {
403+
"max": {
404+
"field": "temperature"
405+
}
406+
}
407+
}
408+
}
409+
```
410+
% TEST[continued]
411+
% TEST[s/_rollup_search/_rollup_search?filter_path=took,timed_out,terminated_early,_shards,hits,aggregations/]
412+
413+
Note the URI now searches `sensor-1` and `sensor_rollup` at the same time.
414+
415+
When the search is executed, the rollup search endpoint does two things:
416+
417+
1. The original request is sent to the non-rollup index unaltered.
418+
2. A rewritten version of the original request is sent to the rollup index.
419+
420+
When the two responses are received, the endpoint rewrites the rollup response
421+
and merges the two together. During the merging process, if there is any overlap
422+
in buckets between the two responses, the buckets from the non-rollup index are
423+
used.
424+
425+
The response to the above query looks as expected, despite spanning rollup and
426+
non-rollup indices:
427+
428+
```console-result
429+
{
430+
"took" : 102,
431+
"timed_out" : false,
432+
"terminated_early" : false,
433+
"_shards" : ... ,
434+
"hits" : {
435+
"total" : {
436+
"value": 0,
437+
"relation": "eq"
438+
},
439+
"max_score" : 0.0,
440+
"hits" : [ ]
441+
},
442+
"aggregations" : {
443+
"max_temperature" : {
444+
"value" : 202.0
445+
}
446+
}
447+
}
448+
```
449+
% TESTRESPONSE[s/"took" : 102/"took" : $body.$_path/]
450+
% TESTRESPONSE[s/"_shards" : \.\.\. /"_shards" : $body.$_path/]

0 commit comments

Comments
 (0)