Skip to content

query.proto revamp#1251

Merged
sougou merged 19 commits into
masterfrom
proto
Oct 28, 2015
Merged

query.proto revamp#1251
sougou merged 19 commits into
masterfrom
proto

Conversation

@sougou
Copy link
Copy Markdown
Contributor

@sougou sougou commented Oct 23, 2015

WIP: DO NOT SUBMIT
This is to allow people to start reviewing work done so far:

  • New Type defined, and support functions done.
  • QueryResult changed to use new structure.
  • Unit tests are passing, by p3 clients are not done yet.
  • BindVars work is not done yet.

This change defines the new types, how they are derived from
MySQL, and tests to make sure the conversions work.
So far, unit tests are passing. Still need to look at clients.
Tiny, Long, etc. are not very understandable. I've now changed the
types to intXX, which is more intuitive.
Comment thread go/sqltypes/type.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took me a long time to understand what you're doing here... ;-).
Can you add a comment describing your method for converting MySQL types to a Vitess Type? (It would be helpful to mention that you first convert to a base type and then modify based on mysql flags)
Also, don't you need to shift by 2 bytes here? Seems your encoding of vitess types takes 2 bytes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Playing with bits is always unreadable. These flags are actually used only against mysql types, and their max val is 255. So, we're safe.
I'm basically trying to build an "a.b" key using field & flags. This is to avoid a 2-level map. I think it's worth the efficiency because this is low level code.
I'll add better comments.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But you're also combining these flags with vitess types which are more than 1 byte long in lines 95--104.
By the way, the explanation that you wrote in your answer to my comment would make a great source code comment ;-).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! Good catch. I'll work on a fix.

I've done enough to get the existing unit tests to pass, but I
think they're insufficient. There also a few TODOs sprayed around.
@sougou
Copy link
Copy Markdown
Contributor Author

sougou commented Oct 24, 2015

Bind vars are now done. I've done enough to get the unit tests passing, but more work is needed.
But this allows others to start fixing the clients. After I finish the left over work, I'll do a best effort to fix the clients, if no one else gets to them.

@sougou
Copy link
Copy Markdown
Contributor Author

sougou commented Oct 26, 2015

All my parts for this branch are done: I've converted all protos to the new vitess type system.
All unit tests and go clients are fixed. I've even fixed the Row part of the java client. What's left:

@alainjobart
Copy link
Copy Markdown
Contributor

LGTM for the proto definitions. Using an enum that is a bit field but has discrete values is a great idea for this. I only look over the code quickly.

I agree we should submit this ASAP. Let's sync up at our team meeting, and see how we can finish this up quickly.

@alainjobart
Copy link
Copy Markdown
Contributor

@erzel @sougou @enisoc
When do you guys want to submit this? We can take php and java being not functional for a couple days if we have to.

@erzel
Copy link
Copy Markdown
Contributor

erzel commented Oct 26, 2015

LGTM. I'd like to gate the mapess code using the new C++ client behind a flag, so I could submit it and we can wait with its activation in prod. So if we're willing to tolerate the open source clients not working for a few days, we can submit now.

@sougou
Copy link
Copy Markdown
Contributor Author

sougou commented Oct 27, 2015

It's a good idea to submit this PR. Maybe we can temporarily disable the Java and PHP tests till they're fixed. That way, we can stay on top of possible new breakages.

@erzel
Copy link
Copy Markdown
Contributor

erzel commented Oct 27, 2015

Submitting this PR would break our import to google3, and I'm currently
busy with submitting my new client changes to google3, so I won't have time
to fix it.
Once I finish with submission of my code, I'll have more time to look at
merging your PR.

Cheers,

Erez.

On Mon, Oct 26, 2015 at 11:02 PM, sougou notifications@github.com wrote:

It's a good idea to submit this PR. Maybe we can temporarily disable the
Java and PHP tests till they're fixed. That way, we can stay on top of
possible new breakages.


Reply to this email directly or view it on GitHub
#1251 (comment).

@michael-berlin
Copy link
Copy Markdown
Contributor

@sougou You can disable both tests in https://github.com/youtube/vitess/blob/master/test/config.json by setting "Manual" to "True".

As we discussed yesterday, disabling the tests for several days in open-source is fine.

@enisoc
Copy link
Copy Markdown
Contributor

enisoc commented Oct 27, 2015

I'm starting work on the PHP client now. @erzel if you become ready to import before I'm done, feel free to merge. If not, I'll add my commits to this PR branch.

@enisoc
Copy link
Copy Markdown
Contributor

enisoc commented Oct 27, 2015

PHP is fixed. I'm going to start fixing the Java client now since no one else has claimed it.

@erzel
Copy link
Copy Markdown
Contributor

erzel commented Oct 28, 2015

The google3 CL is out and awaiting review, hopefully I'll be able to submit it tomorrow. Meanwhile, I'll work on changing the C++ client to the new proto3 structure.
To be clear, I'm fine with merging this PR asap, but I'd like to wait with the import to google3 until the aforementioned CL is submitted.

@enisoc
Copy link
Copy Markdown
Contributor

enisoc commented Oct 28, 2015

Java client fix is ready for review.

@sougou
Copy link
Copy Markdown
Contributor Author

sougou commented Oct 28, 2015

Looks good functionally, and thanks for the Rows cleanup.

sougou added a commit that referenced this pull request Oct 28, 2015
@sougou sougou merged commit 936722a into master Oct 28, 2015
@sougou sougou deleted the proto branch November 3, 2015 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants