Have some sort of HasBuilder type class ?

[nrinaudo]

Is there any chance to have some sort of type class that would provide features similar to:

trait HasBuilder[F[_]] {
  def newBuilder[A]: Builder[A, F[A]]
}

With default instances for all applicable collection types?

I know of at least one concrete use case: type class based decoding libraries. Let's take the example of a very simple CSV decoding library. One would write decoding type class as follows:

// Decodes a single CSV cell
trait CellDecoder[A] {
  def decode(cell: String): Try[A]
}

// Decode an entire CSV row
trait RowDecoder[A] {
  def decode(row: Seq[String]): Try[A]
}

With CanBuildFrom, it's easy to provide an instance of RowDecoder for all collection types:

implicit def cbfDecoder[A, F[_]](implicit da: CellDecoder[A], cbf: CanBuildFrom[Nothing, A, F[A]]): RowDecoder[F[A]] = new RowDecoder[F[A]] {
  override def decode(row: Seq[String]): (row.foldLeft(Success(cbf.apply())) { (acc, cell) => 
    for {
      builder <- acc
      a <- da.decode(cell)
    } yield builder += a).map(_.result())
}

I know of at least three libraries that use this mechanism:

• circe
• argonaut
• kantan.csv

I understand CanBuildFrom is going to disappear, and no mechanism is planned to emulate the above behaviour. The HasBuilder type class I'm suggesting would replace it, and is vastly simpler (and less tied to the core implementation of the collection API) than CBF.

It would ideally be part of the standard collection API, although I can't think of anything that would prevent third party libraries from providing that feature externally.

[nrinaudo]

Oh, and if this or some shape of this were to be made part of the final design, it'd be amazing if it could be Serializable :)

[julienrf]

Hi,

We have something called BuildFrom: https://github.com/scala/collection-strawman/blob/master/src/main/scala/strawman/collection/BuildFrom.scala#L13-L24 It is not yet Serializable, though…

[DarkDimius]

As a general comment about Serializibility, requiring high-level types in standard library to be serializable is both empowering and limiting: it empowers developers of serialization libraries but it limits developers of collections.

I would also argue that java Serializable is a wrong abstraction to start with. The fact that Foo[T] is serializable does not mean that it is in practice serializable for specific T = Any. This entire notion comes from pre-generics java and is very leaky in presence of generics. I believe can do better by using implicit evidences in Scala.

Based on this, I'd be surprised if we would decide to have any of base classes\traits in the collection library extend java Serializable.

[Ichoran]

But that is an argument that the collections themselves shouldn't be serializable, @DarkDimius. Most of the leaf collection classes currently are. Making a provider of a builder be serializable is a substantially less expansive task than making the builder itself be serializable, which in turn is less demanding than making an arbitrary collection be serializable.

[nrinaudo]

@julienrf it's not quite the same thing though. What I have in mind lets you create a builder out of thin air while what you linked requires an existing value. Granted, this addresses the specific example I wrote, but not any example where you get a bunch of values in a type that is not part of the standard collections. Imagine my CSV rows were not sequences but a custom linked list, say.

It seems a cheap abstraction to have, with one clear use case: creating new collections without requiring already having a collection in the first place.

@DarkDimius I agree that in an ideal world, Serializable would not exist. The reason I'm asking is that existing and popular frameworks such as Spark only work with serializable values, and non-serializable type classes often become unusable in that context.

Maybe the base type class shouldn't be, but all implementations should? CBF already tries to do that, I think, it's just not doing a terrible job of it.

[julienrf]

@nrinaudo Yes, you are right. I agree with you that we should have a type-directed-only builder provider.

BTW there is some discussion in this PR #45 (comment)

[DarkDimius]

@Ichoran, I wholeheartedly agree that we should make leaf collection types and builder providers serializable. I am not convinced that we should make the base-trait serializable though.

[nrinaudo]

@DarkDimius valid point. But then, there might need to be some sort of test for this, to avoid situations like CBF where it's pretty random and cannot be relied on. I'd almost rather an API that was honest about not supporting a feature rather than one that does half of the job and leaves it at that.

[julienrf]

Small update: BuildFrom has been removed.

[julienrf]

Fixed by #136.