Skip to content

Commit 5cc17ba

Browse files
c21cloud-fan
authored andcommitted
[SPARK-35351][SQL][FOLLOWUP] Avoid using loaded variable for LEFT ANTI SMJ code-gen
### What changes were proposed in this pull request? This is a followup from #32547 (comment), where for LEFT ANTI join, we do not need to depend on `loaded` variable, as in `codegenAnti` we only load `streamedAfter` no more than once (i.e. assign column values from streamed row which are not used in join condition). ### Why are the changes needed? Avoid unnecessary processing in code-gen (though it's just `boolean $loaded = false;`, and `if (!$loaded) { $loaded = true; }`). ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing unite tests in `ExistenceJoinSuite`. Closes #32681 from c21/join-followup. Authored-by: Cheng Su <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent 79a2a46 commit 5cc17ba

File tree

1 file changed

+17
-14
lines changed

1 file changed

+17
-14
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala

Lines changed: 17 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -681,26 +681,29 @@ case class SortMergeJoinExec(
681681
val cond = BindReferences.bindReference(
682682
condition.get, streamedPlan.output ++ bufferedPlan.output).genCode(ctx)
683683
// Evaluate the columns those used by condition before loop
684-
val before =
685-
s"""
686-
|boolean $loaded = false;
687-
|$streamedBefore
688-
""".stripMargin
689-
690-
val loadStreamed =
691-
s"""
692-
|if (!$loaded) {
693-
| $loaded = true;
694-
| $streamedAfter
695-
|}
684+
val before = joinType match {
685+
case LeftAnti =>
686+
// No need to initialize `loaded` variable for Left Anti join.
687+
streamedBefore.trim
688+
case _ =>
689+
s"""
690+
|boolean $loaded = false;
691+
|$streamedBefore
696692
""".stripMargin
693+
}
697694

698695
val loadStreamedAfterCondition = joinType match {
699696
case LeftAnti =>
700697
// No need to evaluate columns not used by condition from streamed side, as for Left Anti
701698
// join, streamed row with match is not outputted.
702699
""
703-
case _ => loadStreamed
700+
case _ =>
701+
s"""
702+
|if (!$loaded) {
703+
| $loaded = true;
704+
| $streamedAfter
705+
|}
706+
""".stripMargin
704707
}
705708

706709
val loadBufferedAfterCondition = joinType match {
@@ -722,7 +725,7 @@ case class SortMergeJoinExec(
722725
|$loadStreamedAfterCondition
723726
|$loadBufferedAfterCondition
724727
""".stripMargin
725-
(before, checking.trim, loadStreamed)
728+
(before, checking.trim, streamedAfter.trim)
726729
} else {
727730
(evaluateVariables(streamedVars), "", "")
728731
}

0 commit comments

Comments
 (0)