Adding code to properly split_clone a v3 table.#1634
Conversation
We need a vindex of cost < 2 on a table (that is numeric or hash). Then we use it to split the rows.
|
Actually, we should just go for the primary vindex (colvindexes[0]), because it will most likely be the sharding key (lowest cost). It's also guaranteed to be unique, which is another requirement. |
| scw.keyspace: kformal, | ||
| }, | ||
| } | ||
| vschema, err := vindexes.BuildVSchema(formal) |
There was a problem hiding this comment.
I think you already noticed. This code is very similar to vindexes.Validate. We could look at sharing.
I see the rest of the stuff is WIP. I'll comment as things develop.
It turns out it's the most likely one, no need to scan them all.
Building the formal vschema for a keyspace was shared in a couple places.
The test was returning all kinds of rows, had to fix that, because now the filtering can be done at the app layer for v3. Also, the test was describing views in the test schema, which is not right, as the code doesn't ask for them.
We used to think we should run one diff per destination shard. Now we assume we want one diff for each (source, destination) tuple. That way splits and merges will work the same way, and we never need merge-sort. In the process, fix the keyrange overlap usage to work for both splits and merges.
|
@sougou this is ready for review. |
| if colVindex.Vindex.Cost() > 1 { | ||
| return nil, fmt.Errorf("primary vindex cost is too high for table %v", td.Name) | ||
| } | ||
| unique, ok := colVindex.Vindex.(vindexes.Unique) |
There was a problem hiding this comment.
Nit: use vindexes.IsUnique instead.
There was a problem hiding this comment.
Ahh. Nvm. Looks like you need the Unique interface var also.
|
LGTM |
|
LGTM Some minor comments. Feel free to fix them forward if you like. Reviewed 4 of 4 files at r1, 2 of 2 files at r2, 2 of 2 files at r3, 2 of 2 files at r4, 5 of 5 files at r5, 2 of 2 files at r6. go/vt/worker/diff_utils.go, line 84 [r5] (raw file): go/vt/worker/split_diff_cmd.go, line 66 [r6] (raw file): go/vt/worker/split_diff_cmd.go, line 172 [r6] (raw file): go/vt/worker/split_diff_test.go, line 287 [r5] (raw file): go/vt/worker/split_diff_test.go, line 288 [r5] (raw file): Comments from Reviewable |
4 participants
We need a vindex of cost < 2 on a table (that is numeric or hash).
Then we use it to split the rows.
@sougou you can take a look if you want, but I'm gonna keep working on it to add a few more things there.
This change is