Implement jsg::LazyPromise<T> #5878
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the
WritableStreamDefaultWriter, thereadyandclosedpromises end up being always created even if the accessors are never called and nothing is ever paying attention. In cases where there are lots of writes to a stream, this means that manyreadypromises end up being created but never paid attention to and never resolved, just adding overhead.The
jsg::LazyPromise<T>lazily producesjsg::Promise<T>instances, by replacing thereadyandclosedpromises withjsg::LazyPromise<T>we can significantly reduce the number of promises created. The interface still allows the ready/closed to be rejected/resolved while deferring the creation of the actual JS promise and wrapping jsg::Promise until it is actually requested.For example, in the sample introduced in this PR, for the paths that use the TransformStream, there end up being 20 writes to the TransformStream writable side. Because it's a pipe-through, we're not using the typical backpressure signal and we end up with 20 extraneously created ready promises that are wasted allocations. These are visible in the graph below as the orphaned row of promises across the top.
While these 20 promises do not individually add significant overhead on their own, in aggregate that's a lot of extraneous allocation (20x wasted promise allocations per request is non-trivial in aggregate)
The
jsg::LazyPromise<T>is not another promise API... it just will lazily create thejsg::Promise<T>and underlying JS promise only when absolutely necessary to do so while preserving the flow of being able to resolve/reject them when necessary.I have this PR as draft because I'm evaluating the effectiveness of the change. For the pipethrough case above it's not immediately effective because TransformStream's have their own backpressure signal that ends up materializing the promise always anyway, so if the stream is operating fully synchronously we end up hitting backpressure toggles frequently. Async transforms tend to avoid this churn. So it's not yet immediately clear if this is going to have enough of a benefit. Until I've proven one way or the other, I'll leave this as a draft.