[SPARK-4699] [SQL] Make caseSensitive configurable in spark sql analyzer #5806

scwf · 2015-04-30T09:01:29Z

based on #3558

AmplabJenkins · 2015-04-30T09:02:10Z

Merged build triggered.

AmplabJenkins · 2015-04-30T09:02:17Z

Merged build started.

SparkQA · 2015-04-30T09:02:35Z

Test build #31410 has started for PR 5806 at commit d5a9933.

SparkQA · 2015-04-30T10:30:00Z

Test build #31410 has finished for PR 5806 at commit d5a9933.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class Param[T] (val parent: Params, val name: String, val doc: String, val isValid: T => Boolean)
- class DoubleParam(parent: Params, name: String, doc: String, isValid: Double => Boolean)
- class IntParam(parent: Params, name: String, doc: String, isValid: Int => Boolean)
- class FloatParam(parent: Params, name: String, doc: String, isValid: Float => Boolean)
- class LongParam(parent: Params, name: String, doc: String, isValid: Long => Boolean)
- class BooleanParam(parent: Params, name: String, doc: String) // No need for isValid
- case class ParamPair[T](param: Param[T], value: T)
- class KMeansModel (
- trait PMMLExportable
- class SimpleCatalog(val conf: CatalystConf) extends Catalog
This patch adds the following new dependencies:
- jaxb-api-2.2.7.jar
- jaxb-core-2.2.7.jar
- jaxb-impl-2.2.7.jar
- pmml-agent-1.1.15.jar
- pmml-model-1.1.15.jar
- pmml-schema-1.1.15.jar
This patch removes the following dependencies:
- activation-1.1.jar
- jaxb-api-2.2.2.jar
- jaxb-impl-2.2.3-1.jar

AmplabJenkins · 2015-04-30T10:30:04Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-04-30T10:30:04Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31410/
Test FAILed.

AmplabJenkins · 2015-05-08T04:12:20Z

Merged build started.

SparkQA · 2015-05-08T04:14:11Z

Test build #32194 has started for PR 5806 at commit cd51712.

SparkQA · 2015-05-08T04:23:22Z

Test build #32194 has finished for PR 5806 at commit cd51712.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class SimpleCatalystConf(caseSensitiveAnalysis: Boolean) extends CatalystConf
- class SimpleCatalog(val conf: CatalystConf) extends Catalog

AmplabJenkins · 2015-05-08T04:23:25Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-08T04:23:25Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32194/
Test FAILed.

scwf · 2015-05-08T04:30:14Z

retest this please

AmplabJenkins · 2015-05-08T04:32:11Z

Merged build triggered.

AmplabJenkins · 2015-05-08T04:32:21Z

Merged build started.

SparkQA · 2015-05-08T04:33:02Z

Test build #32198 has started for PR 5806 at commit cd51712.

SparkQA · 2015-05-08T06:28:13Z

Test build #32198 has finished for PR 5806 at commit cd51712.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class SimpleCatalystConf(caseSensitiveAnalysis: Boolean) extends CatalystConf
- class SimpleCatalog(val conf: CatalystConf) extends Catalog

AmplabJenkins · 2015-05-08T06:28:17Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-08T06:28:17Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32198/
Test PASSed.

scwf · 2015-05-08T06:31:22Z

sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala

@@ -32,6 +34,7 @@ private[spark] object SQLConf {
  val CODEGEN_ENABLED = "spark.sql.codegen"
  val UNSAFE_ENABLED = "spark.sql.unsafe.enabled"
  val DIALECT = "spark.sql.dialect"
+  val CASE_SENSITIVE = "spark.sql.caseSensitive"


now place this config in SQLConf

scwf · 2015-05-08T06:32:24Z

updated /cc @marmbrus , please take a look

based on #3558 Author: Jacky Li <[email protected]> Author: wangfei <[email protected]> Author: scwf <[email protected]> Closes #5806 from scwf/case and squashes the following commits: cd51712 [wangfei] fix compile d4b724f [wangfei] address michael's comment af512c7 [wangfei] fix conflicts 4ef1be7 [wangfei] fix conflicts 269cf21 [scwf] fix conflicts b73df6c [scwf] style issue 9e11752 [scwf] improve SimpleCatalystConf b35529e [scwf] minor style a3f7659 [scwf] remove unsed imports 2a56515 [scwf] fix conflicts 6db4bf5 [scwf] also fix for HiveContext 7fc4a98 [scwf] fix test case d5a9933 [wangfei] fix style eee75ba [wangfei] fix EmptyConf 6ef31cf [wangfei] revert pom changes 5d7c456 [wangfei] set CASE_SENSITIVE false in TestHive 966e719 [wangfei] set CASE_SENSITIVE false in hivecontext fd30e25 [wangfei] added override 69b3b70 [wangfei] fix AnalysisSuite 5472b08 [wangfei] fix compile issue 56034ca [wangfei] fix conflicts and improve for catalystconf 664d1e9 [Jacky Li] Merge branch 'master' of https://github.com/apache/spark into case 12eca9a [Jacky Li] solve conflict with master 39e369c [Jacky Li] fix confilct after DataFrame PR dee56e9 [Jacky Li] fix test case failure 05b09a3 [Jacky Li] fix conflict base on the latest master branch 73c16b1 [Jacky Li] fix bug in sql/hive 9bf4cc7 [Jacky Li] fix bug in catalyst 005c56d [Jacky Li] make SQLContext caseSensitivity configurable 6332e0f [Jacky Li] fix bug fcbf0d9 [Jacky Li] fix scalastyle check e7bca31 [Jacky Li] make caseSensitive configuration in Analyzer and Catalog 91b1b96 [Jacky Li] make caseSensitive configurable in Analyzer f57f15c [Jacky Li] add testcase 578d167 [Jacky Li] make caseSensitive configurable (cherry picked from commit 6dad76e) Signed-off-by: Michael Armbrust <[email protected]>

evacchi · 2015-05-14T16:03:20Z

Is FunctionRegistry supposed to use this CatalystConf as well ?

scwf · 2015-05-15T01:03:33Z

yes it should use it, thanks i will give a PR today.

follow up for #5806 Author: scwf <[email protected]> Closes #6164 from scwf/FunctionRegistry and squashes the following commits: 15e6697 [scwf] use catalogconf in FunctionRegistry (cherry picked from commit 60336e3) Signed-off-by: Michael Armbrust <[email protected]>

follow up for #5806 Author: scwf <[email protected]> Closes #6164 from scwf/FunctionRegistry and squashes the following commits: 15e6697 [scwf] use catalogconf in FunctionRegistry

based on apache#3558 Author: Jacky Li <[email protected]> Author: wangfei <[email protected]> Author: scwf <[email protected]> Closes apache#5806 from scwf/case and squashes the following commits: cd51712 [wangfei] fix compile d4b724f [wangfei] address michael's comment af512c7 [wangfei] fix conflicts 4ef1be7 [wangfei] fix conflicts 269cf21 [scwf] fix conflicts b73df6c [scwf] style issue 9e11752 [scwf] improve SimpleCatalystConf b35529e [scwf] minor style a3f7659 [scwf] remove unsed imports 2a56515 [scwf] fix conflicts 6db4bf5 [scwf] also fix for HiveContext 7fc4a98 [scwf] fix test case d5a9933 [wangfei] fix style eee75ba [wangfei] fix EmptyConf 6ef31cf [wangfei] revert pom changes 5d7c456 [wangfei] set CASE_SENSITIVE false in TestHive 966e719 [wangfei] set CASE_SENSITIVE false in hivecontext fd30e25 [wangfei] added override 69b3b70 [wangfei] fix AnalysisSuite 5472b08 [wangfei] fix compile issue 56034ca [wangfei] fix conflicts and improve for catalystconf 664d1e9 [Jacky Li] Merge branch 'master' of https://github.com/apache/spark into case 12eca9a [Jacky Li] solve conflict with master 39e369c [Jacky Li] fix confilct after DataFrame PR dee56e9 [Jacky Li] fix test case failure 05b09a3 [Jacky Li] fix conflict base on the latest master branch 73c16b1 [Jacky Li] fix bug in sql/hive 9bf4cc7 [Jacky Li] fix bug in catalyst 005c56d [Jacky Li] make SQLContext caseSensitivity configurable 6332e0f [Jacky Li] fix bug fcbf0d9 [Jacky Li] fix scalastyle check e7bca31 [Jacky Li] make caseSensitive configuration in Analyzer and Catalog 91b1b96 [Jacky Li] make caseSensitive configurable in Analyzer f57f15c [Jacky Li] add testcase 578d167 [Jacky Li] make caseSensitive configurable

follow up for apache#5806 Author: scwf <[email protected]> Closes apache#6164 from scwf/FunctionRegistry and squashes the following commits: 15e6697 [scwf] use catalogconf in FunctionRegistry

based on apache#3558 Author: Jacky Li <[email protected]> Author: wangfei <[email protected]> Author: scwf <[email protected]> Closes apache#5806 from scwf/case and squashes the following commits: cd51712 [wangfei] fix compile d4b724f [wangfei] address michael's comment af512c7 [wangfei] fix conflicts 4ef1be7 [wangfei] fix conflicts 269cf21 [scwf] fix conflicts b73df6c [scwf] style issue 9e11752 [scwf] improve SimpleCatalystConf b35529e [scwf] minor style a3f7659 [scwf] remove unsed imports 2a56515 [scwf] fix conflicts 6db4bf5 [scwf] also fix for HiveContext 7fc4a98 [scwf] fix test case d5a9933 [wangfei] fix style eee75ba [wangfei] fix EmptyConf 6ef31cf [wangfei] revert pom changes 5d7c456 [wangfei] set CASE_SENSITIVE false in TestHive 966e719 [wangfei] set CASE_SENSITIVE false in hivecontext fd30e25 [wangfei] added override 69b3b70 [wangfei] fix AnalysisSuite 5472b08 [wangfei] fix compile issue 56034ca [wangfei] fix conflicts and improve for catalystconf 664d1e9 [Jacky Li] Merge branch 'master' of https://github.com/apache/spark into case 12eca9a [Jacky Li] solve conflict with master 39e369c [Jacky Li] fix confilct after DataFrame PR dee56e9 [Jacky Li] fix test case failure 05b09a3 [Jacky Li] fix conflict base on the latest master branch 73c16b1 [Jacky Li] fix bug in sql/hive 9bf4cc7 [Jacky Li] fix bug in catalyst 005c56d [Jacky Li] make SQLContext caseSensitivity configurable 6332e0f [Jacky Li] fix bug fcbf0d9 [Jacky Li] fix scalastyle check e7bca31 [Jacky Li] make caseSensitive configuration in Analyzer and Catalog 91b1b96 [Jacky Li] make caseSensitive configurable in Analyzer f57f15c [Jacky Li] add testcase 578d167 [Jacky Li] make caseSensitive configurable

follow up for apache#5806 Author: scwf <[email protected]> Closes apache#6164 from scwf/FunctionRegistry and squashes the following commits: 15e6697 [scwf] use catalogconf in FunctionRegistry

based on apache#3558 Author: Jacky Li <[email protected]> Author: wangfei <[email protected]> Author: scwf <[email protected]> Closes apache#5806 from scwf/case and squashes the following commits: cd51712 [wangfei] fix compile d4b724f [wangfei] address michael's comment af512c7 [wangfei] fix conflicts 4ef1be7 [wangfei] fix conflicts 269cf21 [scwf] fix conflicts b73df6c [scwf] style issue 9e11752 [scwf] improve SimpleCatalystConf b35529e [scwf] minor style a3f7659 [scwf] remove unsed imports 2a56515 [scwf] fix conflicts 6db4bf5 [scwf] also fix for HiveContext 7fc4a98 [scwf] fix test case d5a9933 [wangfei] fix style eee75ba [wangfei] fix EmptyConf 6ef31cf [wangfei] revert pom changes 5d7c456 [wangfei] set CASE_SENSITIVE false in TestHive 966e719 [wangfei] set CASE_SENSITIVE false in hivecontext fd30e25 [wangfei] added override 69b3b70 [wangfei] fix AnalysisSuite 5472b08 [wangfei] fix compile issue 56034ca [wangfei] fix conflicts and improve for catalystconf 664d1e9 [Jacky Li] Merge branch 'master' of https://github.com/apache/spark into case 12eca9a [Jacky Li] solve conflict with master 39e369c [Jacky Li] fix confilct after DataFrame PR dee56e9 [Jacky Li] fix test case failure 05b09a3 [Jacky Li] fix conflict base on the latest master branch 73c16b1 [Jacky Li] fix bug in sql/hive 9bf4cc7 [Jacky Li] fix bug in catalyst 005c56d [Jacky Li] make SQLContext caseSensitivity configurable 6332e0f [Jacky Li] fix bug fcbf0d9 [Jacky Li] fix scalastyle check e7bca31 [Jacky Li] make caseSensitive configuration in Analyzer and Catalog 91b1b96 [Jacky Li] make caseSensitive configurable in Analyzer f57f15c [Jacky Li] add testcase 578d167 [Jacky Li] make caseSensitive configurable

follow up for apache#5806 Author: scwf <[email protected]> Closes apache#6164 from scwf/FunctionRegistry and squashes the following commits: 15e6697 [scwf] use catalogconf in FunctionRegistry

Jacky Li and others added 23 commits December 3, 2014 01:33

make caseSensitive configurable

578d167

add testcase

f57f15c

make caseSensitive configurable in Analyzer

91b1b96

make caseSensitive configuration in Analyzer and Catalog

e7bca31

fix scalastyle check

fcbf0d9

fix bug

6332e0f

make SQLContext caseSensitivity configurable

005c56d

fix bug in catalyst

9bf4cc7

fix bug in sql/hive

73c16b1

fix conflict base on the latest master branch

05b09a3

fix test case failure

dee56e9

fix confilct after DataFrame PR

39e369c

solve conflict with master

12eca9a

Merge branch 'master' of https://github.com/apache/spark into case

664d1e9

fix conflicts and improve for catalystconf

56034ca

fix compile issue

5472b08

fix AnalysisSuite

69b3b70

added override

fd30e25

set CASE_SENSITIVE false in hivecontext

966e719

set CASE_SENSITIVE false in TestHive

5d7c456

revert pom changes

6ef31cf

fix EmptyConf

eee75ba

fix style

d5a9933

fix test case

7fc4a98

scwf reviewed May 8, 2015
View reviewed changes

asfgit closed this in 6dad76e May 8, 2015

scwf deleted the case branch May 9, 2015 00:56

scwf mentioned this pull request May 15, 2015

[SPARK-7656] [SQL] use CatalystConf in FunctionRegistry #6164

Closed

JoshRosen mentioned this pull request Aug 1, 2015

[SPARK-4699][SQL] make caseSensitive configurable in Analyzer.scala #3558

Closed

[SPARK-4699] [SQL] Make caseSensitive configurable in spark sql analyzer #5806

[SPARK-4699] [SQL] Make caseSensitive configurable in spark sql analyzer #5806

Uh oh!

Conversation

scwf commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

SparkQA commented Apr 30, 2015

Uh oh!

SparkQA commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

AmplabJenkins commented May 8, 2015

Uh oh!

SparkQA commented May 8, 2015

Uh oh!

SparkQA commented May 8, 2015

Uh oh!

AmplabJenkins commented May 8, 2015

Uh oh!

AmplabJenkins commented May 8, 2015

Uh oh!

scwf commented May 8, 2015

Uh oh!

AmplabJenkins commented May 8, 2015

Uh oh!

AmplabJenkins commented May 8, 2015

Uh oh!

SparkQA commented May 8, 2015

Uh oh!

SparkQA commented May 8, 2015

Uh oh!

AmplabJenkins commented May 8, 2015

Uh oh!

AmplabJenkins commented May 8, 2015

Uh oh!

scwf May 8, 2015

Choose a reason for hiding this comment

Uh oh!

scwf commented May 8, 2015

Uh oh!

evacchi commented May 14, 2015

Uh oh!

scwf commented May 15, 2015

Uh oh!

Uh oh!