Skip to content

Commit 02c3352

Browse files
LuciferYangHyukjinKwon
authored andcommitted
[SPARK-42921][SQL][TESTS] Split timestampNTZ/datetime-special.sql into w/ and w/o ansi suffix to pass sql analyzer test in ansi mode
### What changes were proposed in this pull request? After #40496, run ``` SPARK_ANSI_SQL_MODE=true build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite" ``` There is one test faild with `spark.sql.ansi.enabled = true` ``` [info] - timestampNTZ/datetime-special.sql_analyzer_test *** FAILED *** (11 milliseconds) [info] timestampNTZ/datetime-special.sql_analyzer_test [info] Expected "...date(999999, 3, 18, [false) AS make_date(999999, 3, 18)#x, make_date(-1, 1, 28, fals]e) AS make_date(-1, ...", but got "...date(999999, 3, 18, [true) AS make_date(999999, 3, 18)#x, make_date(-1, 1, 28, tru]e) AS make_date(-1, ..." Result did not match for query #1 [info] select make_date(999999, 3, 18), make_date(-1, 1, 28) (SQLQueryTestSuite.scala:777) [info] org.scalatest.exceptions.TestFailedException: ``` The failure reason is the last parameter of function `MakeDate` is `failOnError: Boolean = SQLConf.get.ansiEnabled`. So this pr split `timestampNTZ/datetime-special.sql` into w/ and w/o ansi to mask this test difference. ### Why are the changes needed? Make SQLQueryTestSuite test pass with `spark.sql.ansi.enabled = true`. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions - Manual checked `SPARK_ANSI_SQL_MODE=true build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite"` Closes #40552 from LuciferYang/SPARK-42921. Authored-by: yangjie01 <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 06bf544 commit 02c3352

File tree

5 files changed

+63
-1
lines changed

5 files changed

+63
-1
lines changed
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
-- Automatically generated by SQLQueryTestSuite
2+
-- !query
3+
select date'999999-03-18', date'-0001-1-28', date'0015'
4+
-- !query analysis
5+
[Analyzer test output redacted due to nondeterminism]
6+
7+
8+
-- !query
9+
select make_date(999999, 3, 18), make_date(-1, 1, 28)
10+
-- !query analysis
11+
Project [make_date(999999, 3, 18, true) AS make_date(999999, 3, 18)#x, make_date(-1, 1, 28, true) AS make_date(-1, 1, 28)#x]
12+
+- OneRowRelation
13+
14+
15+
-- !query
16+
select timestamp'-1969-12-31 16:00:00', timestamp'-0015-03-18 16:00:00', timestamp'-000001', timestamp'99999-03-18T12:03:17'
17+
-- !query analysis
18+
Project [-1969-12-31 16:00:00 AS TIMESTAMP_NTZ '-1969-12-31 16:00:00'#x, -0015-03-18 16:00:00 AS TIMESTAMP_NTZ '-0015-03-18 16:00:00'#x, -0001-01-01 00:00:00 AS TIMESTAMP_NTZ '-0001-01-01 00:00:00'#x, +99999-03-18 12:03:17 AS TIMESTAMP_NTZ '+99999-03-18 12:03:17'#x]
19+
+- OneRowRelation
20+
21+
22+
-- !query
23+
select make_timestamp(-1969, 12, 31, 16, 0, 0.0), make_timestamp(-15, 3, 18, 16, 0, 0.0), make_timestamp(99999, 3, 18, 12, 3, 17.0)
24+
-- !query analysis
25+
Project [make_timestamp(-1969, 12, 31, 16, 0, cast(0.0 as decimal(16,6)), None, Some(America/Los_Angeles), true, TimestampNTZType) AS make_timestamp(-1969, 12, 31, 16, 0, 0.0)#x, make_timestamp(-15, 3, 18, 16, 0, cast(0.0 as decimal(16,6)), None, Some(America/Los_Angeles), true, TimestampNTZType) AS make_timestamp(-15, 3, 18, 16, 0, 0.0)#x, make_timestamp(99999, 3, 18, 12, 3, cast(17.0 as decimal(16,6)), None, Some(America/Los_Angeles), true, TimestampNTZType) AS make_timestamp(99999, 3, 18, 12, 3, 17.0)#x]
26+
+- OneRowRelation
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
--SET spark.sql.ansi.enabled = true
2+
--IMPORT datetime-special.sql
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
1+
--SET spark.sql.ansi.enabled = false
12
--IMPORT datetime-special.sql
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
-- Automatically generated by SQLQueryTestSuite
2+
-- !query
3+
select date'999999-03-18', date'-0001-1-28', date'0015'
4+
-- !query schema
5+
struct<DATE '+999999-03-18':date,DATE '-0001-01-28':date,DATE '0015-01-01':date>
6+
-- !query output
7+
+999999-03-18 -0001-01-28 0015-01-01
8+
9+
10+
-- !query
11+
select make_date(999999, 3, 18), make_date(-1, 1, 28)
12+
-- !query schema
13+
struct<make_date(999999, 3, 18):date,make_date(-1, 1, 28):date>
14+
-- !query output
15+
+999999-03-18 -0001-01-28
16+
17+
18+
-- !query
19+
select timestamp'-1969-12-31 16:00:00', timestamp'-0015-03-18 16:00:00', timestamp'-000001', timestamp'99999-03-18T12:03:17'
20+
-- !query schema
21+
struct<TIMESTAMP_NTZ '-1969-12-31 16:00:00':timestamp_ntz,TIMESTAMP_NTZ '-0015-03-18 16:00:00':timestamp_ntz,TIMESTAMP_NTZ '-0001-01-01 00:00:00':timestamp_ntz,TIMESTAMP_NTZ '+99999-03-18 12:03:17':timestamp_ntz>
22+
-- !query output
23+
-1969-12-31 16:00:00 -0015-03-18 16:00:00 -0001-01-01 00:00:00 +99999-03-18 12:03:17
24+
25+
26+
-- !query
27+
select make_timestamp(-1969, 12, 31, 16, 0, 0.0), make_timestamp(-15, 3, 18, 16, 0, 0.0), make_timestamp(99999, 3, 18, 12, 3, 17.0)
28+
-- !query schema
29+
struct<make_timestamp(-1969, 12, 31, 16, 0, 0.0):timestamp_ntz,make_timestamp(-15, 3, 18, 16, 0, 0.0):timestamp_ntz,make_timestamp(99999, 3, 18, 12, 3, 17.0):timestamp_ntz>
30+
-- !query output
31+
-1969-12-31 16:00:00 -0015-03-18 16:00:00 +99999-03-18 12:03:17

sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerQueryTestSuite.scala

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,9 @@ class ThriftServerQueryTestSuite extends SQLQueryTestSuite with SharedThriftServ
9898
"subquery/in-subquery/in-group-by.sql",
9999
"subquery/in-subquery/simple-in.sql",
100100
"subquery/in-subquery/in-order-by.sql",
101-
"subquery/in-subquery/in-set-operations.sql"
101+
"subquery/in-subquery/in-set-operations.sql",
102+
// SPARK-42921
103+
"timestampNTZ/datetime-special-ansi.sql"
102104
)
103105

104106
override def runQueries(

0 commit comments

Comments
 (0)