Typeless parent/child

[martijnvg]

Related to #15613

Types are going to be removed. The parent/child is tightly coupled to types, so that needs to be changed. Parent/child still needs to distinguish between a parent or a child document, so the idea is that a new meta field type name _join will be allow that. This new field would replace _parent meta field. Each _join meta field type would maintain an indexed field to distinguish between parent and child documents and a doc values field needed for the join operation.

Example of how to use typeless parent/child:

PUT /stackoverflow
{
  "mapping": {
    "_join": ["question-to-answers"]
    }
  }
}

Indexing question document (parent):

PUT stackoverflow/1?join=question-to-answers
{
  "question" : "..."
}

Indexing answer document (child):

PUT stackoverflow/2?join=question-to-answers&parent=1
{
  "answer" : "..."
}

Besides the parent the typeless parent/child will also need a join url parameter. In order to prevent adding more feature specific options to transport and rest layer, meta fields should be completely isolated, from rest to mapping layer. This will result in cleaner code and allows parent/child to be moved to a module.

Adding answer-to-comments join field:

PUT /index1
{
  "mapping": {
    "_join": ["question-to-answers", "answer-to-comments"]
    }
  }
}

Indexing question document (parent):

PUT stackoverflow/1?join=question-to-answers
{
  "question" : "..."
}

Indexing answer document (child):

PUT stackoverflow/2?join=question-to-answers,answer-to-comments&parent=1
{
  "answer" : "..."
}

Indexing comment document (grand child):

PUT stackoverflow/3?join=answer-to-comments&parent=2&routing=1
{
  "comment" : "..."
}

[jpountz]

I'm not sure we need to retain the notion of types, maybe we could go with something like below? You say that parent_type is used for the name of the internal join field, bu we could directly use the name of the join field directly for that? (in that case question-to-answers)

PUT /stackoverflow
{
  "mappings": {
    "properties": {
      "question-to-answers": {
        "type": "join"
      }
    }
  }
}

PUT stackoverflow/1
{
  "question" : "...",
  "question-to-answers" : null # no parent id, so this doc is a parent
}

PUT stackoverflow/2?routing=1
{
  "answer" : "...",
  "question-to-answers" : "1" # id of the parent
}

Something else that made me wonder when reading your proposal is that users generally do not like modifying the structure of their documents for specifying metadata (about the join in that case), so maybe it should remain a meta field, something like below?

PUT /stackoverflow
{
  "mappings": {
    "_join": [
      "question-to-answers"
    ]
  }
}

PUT stackoverflow/1?join=question-to-answers
{
  "question" : "..."
}

PUT stackoverflow/2?join=question-to-answers&parent=1
{
  "answer" : "..."
}

[martijnvg]

I'm not sure we need to retain the notion of types, maybe we could go with something like below?

When thinking about this refactoring I thought we needed to distinguish between the different documents (in case of multiple join fields), but when answering this question I realize we don't :) ,
because all documents in the same index will have different ids. (currently that is not the case as ids are only unique within a single type)

Something else that made me wonder when reading your proposal is that users generally do not like modifying the structure of their documents for specifying metadata (about the join in that case)

The other reason I moved away from metadata is that besides the parent there is a need of an additional meta field. I didn't want to add more options to IndexRequest and other classes for a feature that isn't used as much as our core features. This gives us cleaner code (p/c's code being encapsulated in only a FieldMapper impl and QueryBuilder impls) and does give the opportunity to eventually isolate parent/child into a module. The price of this is that the relationship needs to be specified in the source of the document.

If we can make metadata completely pluggable (from rest layer to field mapper layer) then we specify the required parameter in the url and having the ability to isolate the p/c code. But I don't feel that this should be a requirement for this refactoring. Although if we really want this we do have the time to develop this.

[jpountz]

I didn't want to add more options to IndexRequest and other classes for a feature that isn't used as much as our core features.

This is a very good point.

[rjernst]

If we can make metadata completely pluggable (from rest layer to field mapper layer) then we specify the required parameter in the url

I think we should do this. It is already a deficiency in the pluggability of Metadata fields. And I like @jpountz suggested api.

[martijnvg]

@rjernst @jpountz I've updated the description of the issue to match with Adrien's proposal.

[stephanebastian]

Hi All,

Interesting discussion, I'm jumping-in with a couple of questions/comments.

Right now, as a user I can independently define the properties of my parent and child documents.
If I understand correctly, one consequence of removing types, is that I would define ONE mapping defining all properties of both my parent and child document (basically a merge of my current child and parent mapping files). Is that right?

Then, when inserting a document, I would either set properties allowed by the parent OR properties allowed by the child. ES won't have any way to tell if the properties are 'parent properties' or 'child properties'. Am I correct ?
If that's right, it seems like a major step backward in terms of functionality. What do you guys think?

Another question: Would it be possible to support cross index parent-child relationships?

All the best

[jpountz]

Your assumptions are correct. It is indeed hard to keep feature parity when moving forward, however we think that removing types remains a good trade-off by making Elasticsearch easier to use, easier to understand and possibly faster.

Cross index parent-child relationships would not be possible as-is since Elasticsearch relies on the fact that a parent and all its children are on the same shard.