Skip to content

[SPARK-6618][SPARK-6669][SQL] Lock Hive metastore client correctly. #5333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 7 commits into from
Closed

[SPARK-6618][SPARK-6669][SQL] Lock Hive metastore client correctly. #5333

wants to merge 7 commits into from

Conversation

yhuai
Copy link
Contributor

@yhuai yhuai commented Apr 2, 2015

No description provided.

@yhuai
Copy link
Contributor Author

yhuai commented Apr 2, 2015

I am working on the test.

@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29617 has started for PR 5333 at commit 1e241af.

@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29617 has finished for PR 5333 at commit 1e241af.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
  • This patch does not change any dependencies.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29617/
Test FAILed.

marmbrus
marmbrus reviewed Apr 2, 2015
View reviewed changes
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
@@ -50,7 +50,7 @@ case class InsertIntoHiveTable(
@transient val sc: HiveContext = sqlContext.asInstanceOf[HiveContext]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Existing: Calling a HiveContext sc is pretty confusing. Also since this is a convenience cast it should be a def so as to not take up extra space in the object.

@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29619 has started for PR 5333 at commit d6fc32f.

@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29619 has finished for PR 5333 at commit d6fc32f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
  • This patch does not change any dependencies.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29619/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29626 has started for PR 5333 at commit 7667030.

@yhuai yhuai changed the title [WIP][SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock [SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock Apr 2, 2015
@yhuai yhuai changed the title [SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock [SPARK-6618][SPARK-6669][SQL] Lock Hive metastore client correctly. Apr 2, 2015
@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29627 has started for PR 5333 at commit 59c884f.

@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29626 has finished for PR 5333 at commit 7667030.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
  • This patch does not change any dependencies.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29626/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Apr 2, 2015

Test build #29627 has finished for PR 5333 at commit 59c884f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
  • This patch does not change any dependencies.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29627/
Test PASSed.

asfgit pushed a commit that referenced this pull request Apr 2, 2015
@yhuai @marmbrus
[SPARK-6618][SPARK-6669][SQL] Lock Hive metastore client correctly.
e6ee95c 
Author: Yin Huai <[email protected]>
Author: Michael Armbrust <[email protected]>

Closes #5333 from yhuai/lookupRelationLock and squashes the following commits:

59c884f [Michael Armbrust] [SQL] Lock metastore client in analyzeTable
7667030 [Yin Huai] Merge pull request #2 from marmbrus/pr/5333
e4a9b0b [Michael Armbrust] Correctly lock on MetastoreCatalog
d6fc32f [Yin Huai] Missing `)`.
1e241af [Yin Huai] Protect InsertIntoHive.
fee7e9c [Yin Huai] A test?
5416b0f [Yin Huai] Just protect client.

(cherry picked from commit 5db8912)
Signed-off-by: Michael Armbrust <[email protected]>
@asfgit asfgit closed this in 5db8912 Apr 2, 2015
@yhuai yhuai deleted the lookupRelationLock branch April 3, 2015 02:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yhuai @SparkQA @AmplabJenkins @marmbrus