Description
Is your feature request related to a problem?
Doing operations with decimal types is currently very slow, since pandas stores them as python Decimal objects. I think it would be nice to have a high-performance, vectorized DecimalArray ExtensionArray type. Would you be open to a PR adding a type like this, or is this something you think belongs outside the pandas library?
Describe the solution you'd like
Create an ExtensionArray datatype that internally represents numbers similar to in pyarrow (as ints, with precision and scale). (Require that all values have the same precision and scale.) Implement basic arithmetic operations (+, -, *, /, etc.).
API breaking implications
By adding a __from_arrow__
and __to_arrow__
field to DecimalArray, this would cause decimal data from pyarrow to be converted to DecimalArray rather than object array by default, which would be a breaking change.
Describe alternatives you've considered
The status quo (loading as object arrays) works, but is slow.
Additional context
This would also be really helpful for loading parquet files that decimal types via pyarrow more quickly, since the step to instantiate python Decimal objects for each value can be very slow for large DataFrames.