Multi column indexes are only used if all columns share operation

[erizocosmico]

Because our methods for Index accept ...interface{}, where the len of that slice is the length of columns in the index, all those methods require all the values, one for each column.

This creates a problem: all columns must use the exact same operation.

For example, consider we have an index on A and B:

• A = 1 AND B = 1 will use the index.
• A > 1 AND B > 5 will use the index.
• A = 1 AND B < 5 will not, because = and < are not the same operation.

[erizocosmico]

Closing, as discussed via slack with @ajnavarro this is expected behaviour.

[ajnavarro]

maybe we should have a look of how is working on mysql or postgres: https://dev.mysql.com/doc/refman/8.0/en/multiple-column-indexes.html

[erizocosmico]

In MySQL, it uses the index as long as the value of the leftmost column is specified.

e.g., for an index on (a, b), it would be used for a = 1 AND b > 5

[erizocosmico]

What are we going to do with this in the end @ajnavarro?

[ajnavarro]

Let's keep that issue open until we find a good solution for that. Right now is not a high priority.

[kuba--]

What if we do not have indexes on multiple columns (internally), so the index on (A, B) would be the same as index on A and index on B.
Ultimately we have to merge 2 indexes (what is pretty fast). Last but not least, it will also dedup. indexes. Right now if you create an index on A, B, C, (A, B), (A, B, C) it means 5 indexes, but if we treat them independently it will be still just 3 indexes (1 per column).
WDYT?

[erizocosmico]

I think that might work. I can't think of any reason this may be bad right now. In fact, it should actually help with updates (less indexes to update once repos change). Only downside I can see is the fact that you will have to check all remaining indexes to see if you can delete a pilosa index or not, because it may be in use in another gitbase index.

WDYT @src-d/data-retrieval?

[ajnavarro]

If it is internally, at pilosa implementation level, I'm totally in on that.

We should save on pilosa metadata the pilosa indexes that go-mysql-server indexes are using to know if we can delete the pilosa index on a DELETE INDEX statement or not.

[kuba--]

Ok, so I'll start prototype the idea of dedup indexes (#261 (comment))