Expected Behavior
When using the || operator (or concat) on a decimal array with a decimal element, the result type should be array(decimal(...)) and values should retain their exact decimal precision.
Current Behavior
With the native sidecar enabled, queries like:
SELECT ARRAY[DECIMAL '1.0'] || DECIMAL '2.0'
return array(double) instead of array(decimal). Values are silently coerced to floating-point, losing decimal precision.
The issue is intermittent — it only surfaces when the native sidecar is active and the expression is structured in a way the optimizer considers constant-foldable, which is why it appears as a flaky failure in TestTpchDistributedQueries#testCoercions.
Possible Solution
Two things need to be addressed:
1. NativeSidecarFunctionRegistryTool exposes native concat signatures to the coordinator
The native worker registers a concat(array(double), double) → array(double) overload. When the coordinator resolves function overloads, this can win over Presto's built-in decimal-aware concat, causing the planned output type to be array(double). Filtering out native concat signatures from getWorkerFunctions() keeps Presto's built-in concat authoritative.
2. NativeExpressionOptimizer passes decimal expressions to the sidecar for constant folding
The native sidecar's wire protocol serializes DECIMAL constants as floating-point. Any decimal sub-expression that gets constant-folded by the sidecar comes back as a DOUBLE. Adding a containsDecimalType guard in visitCall prevents these expressions from being marked constant-foldable, keeping them in Presto's own evaluator.
Steps to Reproduce
- Start Presto with the native sidecar enabled (
sidecarEnabled=true)
- Run a query using decimal array concatenation:
SELECT ARRAY[DECIMAL '1.0'] || DECIMAL '2.0'
- Observe result type is
array(double) and values lose decimal precision
Alternatively, run TestTpchDistributedQueries#testCoercions with sidecar enabled — it will fail intermittently on the decimal coercion cases.
Context
Caught as a flaky failure in testCoercions in our internal CI when running with sidecar enabled. The root cause is two independent paths that both end up coercing decimal to double, neither of which is obvious without tracing through function resolution and the constant-folding optimizer together.
Expected Behavior
When using the
||operator (orconcat) on a decimal array with a decimal element, the result type should bearray(decimal(...))and values should retain their exact decimal precision.Current Behavior
With the native sidecar enabled, queries like:
return
array(double)instead ofarray(decimal). Values are silently coerced to floating-point, losing decimal precision.The issue is intermittent — it only surfaces when the native sidecar is active and the expression is structured in a way the optimizer considers constant-foldable, which is why it appears as a flaky failure in
TestTpchDistributedQueries#testCoercions.Possible Solution
Two things need to be addressed:
1.
NativeSidecarFunctionRegistryToolexposes nativeconcatsignatures to the coordinatorThe native worker registers a
concat(array(double), double) → array(double)overload. When the coordinator resolves function overloads, this can win over Presto's built-in decimal-awareconcat, causing the planned output type to bearray(double). Filtering out nativeconcatsignatures fromgetWorkerFunctions()keeps Presto's built-inconcatauthoritative.2.
NativeExpressionOptimizerpasses decimal expressions to the sidecar for constant foldingThe native sidecar's wire protocol serializes
DECIMALconstants as floating-point. Any decimal sub-expression that gets constant-folded by the sidecar comes back as aDOUBLE. Adding acontainsDecimalTypeguard invisitCallprevents these expressions from being marked constant-foldable, keeping them in Presto's own evaluator.Steps to Reproduce
sidecarEnabled=true)array(double)and values lose decimal precisionAlternatively, run
TestTpchDistributedQueries#testCoercionswith sidecar enabled — it will fail intermittently on the decimal coercion cases.Context
Caught as a flaky failure in
testCoercionsin our internal CI when running with sidecar enabled. The root cause is two independent paths that both end up coercing decimal to double, neither of which is obvious without tracing through function resolution and the constant-folding optimizer together.