Roadmap

[tkowalcz]

Hi,

what are the plans for parallel version of RxJava? As I understand it is intended to lift the restriction on one thread being able to call onNext and thus allow processing the operator chain in parallel.

I would happily contribute but there is no code in repository :) How can I help?

Although I am writing for myself and my opinions are my own I can say that while working at voluum.com where we process large(ish) amounts of data we are finding more and more use cases where we would like to maximize performance and apply parallel processing through observable chain (without synchronization anywhere). We still use RxJava the "traditional" way and we are very happy with it (I cannot stress that enough).

Cheers,
Tomasz

[kbahr]

Hello Tomasz,

We at Netflix also have internal, performance based use cases which is the primary motivation for the project. I volunteered to lead the first cut of the effort as time allowed and unfortunately time has not allowed thus far :)

It's very helpful to have input from another beneficiary, providing additional motivation for the work. I'm putting together the skeleton of the project at the moment and what would also be very helpful is to get your input on use cases for operators. The preliminary set of operators we're targeting are merge, map, flatMap, filter, skip, skipWhile, take, takeWhile, and groupBy. Are there others that you think are high priority/value?

Once the skeleton is pushed out there, in which I would like to include merge and map, the next steps would be fleshing out the other operators for which any help would be welcome.

Thanks,
--Kyle

[tkowalcz]

Hello!

TL;DR: I'll ask around the team, but the ones you listed should suffice (especially merge and groupBy).

Our use case involves distributed analytics service which uses RxJava to asynchronously discover and process files matching users query (from many data sources with varying latency characteristics like S3, SSDs and memory).
This approach scaled from processing 100s of events to 10s of thousands of events per query (sub second). We are starting to reach it limits around 100k, synchronizations (flatMap, merge) costs are getting bigger (but still not dominating the profile), but to be honest newest version we benchmarked is 0.17.2. We will soon move to 0.20.
We already use very little shared mutable state in filters, maps etc. it would be nice to not pay for synchronization on e.g. stream merge points.

I'll wait for the skeleton then :)
Cheers,
Tomek