`tablecloth.api/percentiles` is not supposed to add a column

[daslu]

The following was reported by @lvh on a Slack thread:
https://clojurians.slack.com/archives/C0BQDEJ8M/p1727016052908239

tablecloth.api/percentiles is currently expected to add a column to a dataset, but this does not make sense.
https://github.com/scicloj/tablecloth/blob/1d501aa/src/tablecloth/api.clj#L1790

The percentiles computation indeed returns a collection of numbers, but the elements of that collection are simply the percentiles -- they do not correspond to the actual rows of the dataset, and thus they do not make sense as a new column.

[genmeblog]

It was (accidentally) added by a column project. Here is the full list, maybe it's good to review it again to avoid similar pitfalls.

tablecloth/src/tablecloth/api/api_template.clj

Line 136 in 1d501aa

* + - / < <= > >= abs acos and asin atan atan2 bit-and bit-and-not bit-clear bit-flip bit-not bit-or bit-set bit-shift-left bit-shift-right bit-xor cbrt ceil cos cosh cummax cummin cumprod cumsum distance distance-squared dot-product eq even? exp expm1 finite? floor get-significand hypot identity ieee-remainder infinite? kurtosis log log10 log1p logistic magnitude magnitude-squared mathematical-integer? max mean mean-fast median min nan? neg? next-down next-up normalize not not-eq odd? or percentiles pos? pow quartile-1 quartile-3 quot reduce-* reduce-+ reduce-max reduce-min rem rint round shift signum sin sinh skew sq sqrt sum sum-fast tan tanh to-degrees to-radians ulp unsigned-bit-shift-right variance zero?)

[lvh]

[joinr]

https://www.reddit.com/r/Clojure/comments/1lvxmi6/clojure_tablecloath_percentiles/
percentiles is baffling.

My hack:

(def the-data (->> (for [k [:a :b :c :d]]
                     (let [n (rand-int 10)]
                       [k (repeatedly 100 #(rand-int n))]))
                   (into {})
                   tc/dataset))

(defn simple-percentiles
  "Given a dataset - ds, a collection of column names - cols,
   and an optional collection of percentiles in the range (0 100],
   compute a new dataset with records
   {:column col :p1 p1 :p2 p2 :p3 p3... :pn pn} for each col in cols, p_n in
   percentiles.
   percentiles default to [25 50 75 100]"
  [ds cols & {:keys [percentiles]
              :or {percentiles [25 50 75 100]}}]
  (let [pkeys (map (comp keyword str) percentiles)]
    (->> (for [k cols]
           (merge {:column k}
                  (zipmap pkeys
                          (tech.v3.datatype.statistics/percentiles
                           (ds k) percentiles))))
         tc/dataset)))

user=> (simple-percentiles the-data [:a :b :c :d] :percentiles [1 25 75 100])
_unnamed [4 5]:

| :column |  :1 | :25 | :75 | :100 |
|---------|----:|----:|----:|-----:|
|      :a | 0.0 | 2.0 | 6.0 |  7.0 |
|      :b | 0.0 | 1.0 | 4.0 |  6.0 |
|      :c | 0.0 | 1.0 | 4.0 |  5.0 |
|      :d | 0.0 | 1.0 | 5.0 |  7.0 |