Should we parse search requests on the coordinating node?

[clintongormley]

At the moment, the coordinating node uses the index name from URL and routing values from the query string to forward search requests directly to the shards where they should be executed.

This is very efficient, but does impose some limitations on the choices we can make at execution time. For instance:

• Allow per index routing values in the same query #4272 asks for per-index routing values, which would need to be passed in the body
• Support querying more indices than 4096 bytes-worth #7298 raises the problem of limited space for index names in the URL
• nicer SearchPhaseExecutionException response message for parsing problems #7570 complains about parsing errors being returned from all shards, rather than just one
• Add phase to prefetch doc for query #6719 would allow terms-lookup filters, geo-shape lookup filters, indexed scripts or indexed templates to be fetched just once, instead of by every shard

All of the above (and possibly more?) could be fixed if we parsed the search request in the body. Should we consider doing this? What are the downsides? Would it be worth making it optional?

[martijnvg]

Also the way percolating an existing document works can then be changed, so that the index, type and id can then be defined in the request body instead of the url.

[rjernst]

@clintongormley What do you mean by "making it optional"?

This sounds like a good improvement to me. We should preprocess anything we can on the coordinating node. Especially the issue with getting lots of shard errors instead of one clear error is a personal annoyance.

[clintongormley]

@rjernst i think the reason that we don't do it today is that parsing the JSON creates a bunch of objects which, usually, are not needed. Hence my suggestion of making this optional.

[s1monw]

we discussed this today and we all agreed that this would be awesome to have.

Would it be worth making it optional?

I don't think we should make this optional but it's hard to do everything on the coordinating node. For instance we don't have the mappings instantiated for all indices or maybe not at all. But I think we can take a lot of advantage of this if we do a pre-parsing / rewrite stage. I think this would be awesome to have for lots of stuff like error reporting, prefetching, reducers and cacheing. maybe we should decouple parsing of the query and processing of it! I am +1 on that for sure

[javanna]

I removed the discuss label, I think we have consensus here. We are working on making this possible in #10217 , more specifically on https://github.com/elastic/elasticsearch/tree/feature/query-parse-refactoring branch