Skip to content

Commit c686b7d

Browse files
ueshinmarmbrus
authored andcommitted
[SPARK-2968][SQL] Fix nullabilities of Explode.
Output nullabilities of `Explode` could be detemined by `ArrayType.containsNull` or `MapType.valueContainsNull`. Author: Takuya UESHIN <[email protected]> Closes #1888 from ueshin/issues/SPARK-2968 and squashes the following commits: d128c95 [Takuya UESHIN] Fix nullability of Explode.
1 parent c9c89c3 commit c686b7d

File tree

1 file changed

+4
-4
lines changed
  • sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions

1 file changed

+4
-4
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,19 +86,19 @@ case class Explode(attributeNames: Seq[String], child: Expression)
8686
(child.dataType.isInstanceOf[ArrayType] || child.dataType.isInstanceOf[MapType])
8787

8888
private lazy val elementTypes = child.dataType match {
89-
case ArrayType(et, _) => et :: Nil
90-
case MapType(kt,vt, _) => kt :: vt :: Nil
89+
case ArrayType(et, containsNull) => (et, containsNull) :: Nil
90+
case MapType(kt, vt, valueContainsNull) => (kt, false) :: (vt, valueContainsNull) :: Nil
9191
}
9292

9393
// TODO: Move this pattern into Generator.
9494
protected def makeOutput() =
9595
if (attributeNames.size == elementTypes.size) {
9696
attributeNames.zip(elementTypes).map {
97-
case (n, t) => AttributeReference(n, t, nullable = true)()
97+
case (n, (t, nullable)) => AttributeReference(n, t, nullable)()
9898
}
9999
} else {
100100
elementTypes.zipWithIndex.map {
101-
case (t, i) => AttributeReference(s"c_$i", t, nullable = true)()
101+
case ((t, nullable), i) => AttributeReference(s"c_$i", t, nullable)()
102102
}
103103
}
104104

0 commit comments

Comments
 (0)