Skip to content

[native-sidecar]: Native sidecar silently coerces decimal array concatenation to array(double) #27749

@pramodsatya

Description

@pramodsatya

Expected Behavior

When using the || operator (or concat) on a decimal array with a decimal element, the result type should be array(decimal(...)) and values should retain their exact decimal precision.

Current Behavior

With the native sidecar enabled, queries like:

SELECT ARRAY[DECIMAL '1.0'] || DECIMAL '2.0'

return array(double) instead of array(decimal). Values are silently coerced to floating-point, losing decimal precision.

The issue is intermittent — it only surfaces when the native sidecar is active and the expression is structured in a way the optimizer considers constant-foldable, which is why it appears as a flaky failure in TestTpchDistributedQueries#testCoercions.

Possible Solution

Two things need to be addressed:

1. NativeSidecarFunctionRegistryTool exposes native concat signatures to the coordinator

The native worker registers a concat(array(double), double) → array(double) overload. When the coordinator resolves function overloads, this can win over Presto's built-in decimal-aware concat, causing the planned output type to be array(double). Filtering out native concat signatures from getWorkerFunctions() keeps Presto's built-in concat authoritative.

2. NativeExpressionOptimizer passes decimal expressions to the sidecar for constant folding

The native sidecar's wire protocol serializes DECIMAL constants as floating-point. Any decimal sub-expression that gets constant-folded by the sidecar comes back as a DOUBLE. Adding a containsDecimalType guard in visitCall prevents these expressions from being marked constant-foldable, keeping them in Presto's own evaluator.

Steps to Reproduce

  1. Start Presto with the native sidecar enabled (sidecarEnabled=true)
  2. Run a query using decimal array concatenation:
SELECT ARRAY[DECIMAL '1.0'] || DECIMAL '2.0'
  1. Observe result type is array(double) and values lose decimal precision

Alternatively, run TestTpchDistributedQueries#testCoercions with sidecar enabled — it will fail intermittently on the decimal coercion cases.

Context

Caught as a flaky failure in testCoercions in our internal CI when running with sidecar enabled. The root cause is two independent paths that both end up coercing decimal to double, neither of which is obvious without tracing through function resolution and the constant-folding optimizer together.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

Status
🆕 Unprioritized

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions