How to turn a stream into a signal?

[GDX64]

I want to use the MutableVec to calculate, lets say, the sum of a Vector whenever something changes, to do that It seems one way would be to turn the MutableVec into a stream and them use the scan operator on it, but after that I don't want to be notified of every sum change whitout losses, I want it to behave like a signal again and get the changes only when it is polled. It would be kind of turning the stream result into a signal again. What would be the right way of doing it?

[Pauan]

You can use from_stream to convert a Stream into a Signal.

However, I don't recommend that. Instead, you should use to_signal_map which does exactly what you want:

mutable_vec.signal_vec_cloned().to_signal_map(|items| {
    // items is a &[T] so you can use all of the slice and Iterator methods
    items.into_iter().sum()
})

mutable_vec.signal_vec_cloned().to_signal_map(|items| {
    // You can return a Vec<T> by using the collect method
    items.into_iter()
        .scan(0, |x, y| ...)
        .map(|x| ...)
        .filter(|x| ...)
        .collect::<Vec<_>>()
})

to_signal_map accepts a SignalVec as an input and returns a Signal as the output.

Whenever the SignalVec changes, it will re-run the to_signal_map.

The items is a &[T] which contains all of the current items in the SignalVec.

Whatever the closure returns is the value for the output Signal.

So this allows you to convert a SignalVec into a Signal, while also doing some processing on the items of the SignalVec.

Note that if you want to get the len() of the SignalVec then it's much faster to use the len method instead of to_signal_map.

[GDX64]

Ok, but I wanted to compute the sum incrementally, so that I don't need to iterate through the whole array every time. So I thought about using the stream scan. Would that be the correct way in this case?

[Pauan]

In order to make something incremental, it needs to be designed to use VecDiff, which means it has to be a method on SignalVecExt. Streams will not work, they will not be incremental.

There is already a sum method, so you can use that to get the sum, though it's basically the same as using to_signal_map.

You can't really make sum incremental, because an item can change at any index, so it must iterate over all the items.

[GDX64]

I can, if I know the current state of the vector and an item changes at a given index, I can subtract that old item and add the new one before updating the current state of the vector. The sum was just an example, what I really want is an incremental segment tree.

But you think that the correct way would be extending the vector functionality?

[Pauan]

I can subtract that old item and add the new one before updating the current state of the vector

That would change the type signature, so it wouldn't be compatible with Iterator::sum anymore. That's fine, but it is a change to the public API.

Also, you would still need to hold all of the items for the SignalVec, because VecDiff doesn't give you the old value.

And because you need to store all of the old items, that means every update is O(n) time, not O(1) time like you want.

Also VecDiff::Replace will still require a full O(n) sum.

So the performance might not be any better than to_signal_map.

The sum was just an example, what I really want is an incremental segment tree.

Well that's a completely different issue. SignalVec is for flat Vec-like data structures, not trees.

Perhaps you could explain more about your use case and what you're trying to do?

[GDX64]

Well, I wanted to do this: a chart from a vector that could have potentially some millions of elements. The chart would agregate the data in maximum
and minimum points when it is not very helpful to see each point (as they could be too many). It would be very helpful if I could do it incrementally, because finding maximum and minimum of a range of millions of elements, and scaling, and plotting all of that stuff is not very easy to do in less than 16ms if I can't reuse some calculations. So I wanted to use some kind of reactive programming.

I was thinking about doing something like this with your library:
Keep all the base data in a MutableVec, that mutableVec sends updates of what changed in the vec to somewhere where I can update the segment tree and turn it into a signal, so that the following things that need the data would react to changes in the segment tree. The other things would also need to be signals (like the range I'm currently interested to see on the plot, colors, and so on).

I also played around with some other signal libraries, like leptos and Anchors, but I'm finding it very hard to use those signals efficiently in combination with collections, because when the collection changes, the signal fires and the computations that are listening to those don't know what changed, so they need to recompute lots of things from the collection that a lot of the times they would not really need.

[GDX64]

Also it looks like on rust there is always a lot of cloning involved. The libraries require the data to be clone, which makes it tricky to know when a cloning is going to happen, I makes me uneasy to work with structs that take a lot of memory space. Also when I update the signal it looks like I need to replace it, I can't mutate the data inside (not in leptos, though).

[Pauan]

a chart from a vector that could have potentially some millions of elements. The chart would agregate the data in maximum and minimum points when it is not very helpful to see each point (as they could be too many).

Thanks for the explanation, I've made charts before with FRP, that is possible, and it can be done very efficiently.

Aggregation is difficult to do incrementally, so you'll likely need to have an algorithm to assign each element into a bucket.

For example, if the chart is ordered based on the date, you could assign each data point into buckets based on their day, week, or month.

There isn't any method in SignalVec to do that, but you can write your own method.

finding maximum and minimum of a range of millions of elements, and scaling, and plotting all of that stuff is not very easy to do in less than 16ms if I can't reuse some calculations

It depends on how much data you have. Rust is incredibly fast, and looping over a &[T] is also incredibly fast. So you might find that it's faster and easier to just recalculate the min/max from scratch.

When performance is important, you should always benchmark instead of guessing. Especially with such a fast language like Rust.

Keep all the base data in a MutableVec, that mutableVec sends updates of what changed in the vec to somewhere where I can update the segment tree and turn it into a signal, so that the following things that need the data would react to changes in the segment tree.

Although that is possible, you might have an easier time if you create a custom struct that handles the updates.

Also, segment trees cannot be mutated, so you'd need to recreate the entire tree from scratch on every change.

If you keep the MutableVec in sorted order (which is very easy to do), then you don't need a segment tree anymore, and it becomes much easier and faster to bucket the values.

when the collection changes, the signal fires and the computations that are listening to those don't know what changed, so they need to recompute lots of things from the collection that a lot of the times they would not really need.

Yes, most FRP libraries only provide simple Signals, but futures-signals has excellent collection Signals like SignalVec and SignalMap.

You can also create your own custom Signals, for example a SignalTree.

The libraries require the data to be clone, which makes it tricky to know when a cloning is going to happen.

That's why futures-signals always adds the _cloned suffix for methods which clone.

It's very easy to avoid the cost of cloning: just wrap your struct in Rc or Arc. Now cloning just creates a new pointer, so it's super super super fast. This is common practice.

You can still mutate Mutable and MutableVec even if they are inside of an Rc or Arc (similar to RefCell or Mutex or RwLock), so everything works very nicely.

Also when I update the signal it looks like I need to replace it, I can't mutate the data inside (not in leptos, though).

That's not the case, you can use lock_mut to get a &mut reference to the contents of the Mutable, so you can mutate the data without replacing it.

[GDX64]

That's all valuable information. Thank you very much. Just one more thing. Is there a way to poll synchronously the value of a signal? Just look at it's current value without using the foreach method.

[Pauan]

Is there a way to poll synchronously the value of a signal? Just look at it's current value without using the foreach method.

That's almost always a bad idea. The point of signals is that they give you reactivity, getting the current value breaks reactivity. There's almost certainly a better way of accomplishing your goal without getting the current value.

Having said that, you can convert the Signal into a Stream and then retrieve the next value:

let mut stream = signal.to_stream();

let value1 = stream.next().await;
let value2 = stream.next().await;
let value3 = stream.next().await;