Skip to content

Deprecate mplex #661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Deprecate mplex #661

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 49 additions & 13 deletions mplex/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

> The spec for the friendly Stream Multiplexer (that works in 3 languages!)

| Lifecycle Stage | Maturity | Status | Latest Revision |
|-----------------|----------------|--------|-----------------|
| 3A | Recommendation | Active | r0, 2018-10-10 |
| Lifecycle Stage | Maturity | Status | Latest Revision |
|-----------------|----------------|------------|-----------------|
| 3A | Recommendation | Deprecated | r0, 2018-10-10 |

Authors: [@daviddias], [@Stebalien], [@tomaka]

Expand All @@ -25,16 +25,17 @@ and spec status.
## Table of Contents

- [mplex](#mplex)
- [Table of Contents](#table-of-contents)
- [Overview](#overview)
- [Message format](#message-format)
- [Flag Values](#flag-values)
- [Protocol](#protocol)
- [Opening a new stream](#opening-a-new-stream)
- [Writing to a stream](#writing-to-a-stream)
- [Closing a stream](#closing-a-stream)
- [Resetting a stream](#resetting-a-stream)
- [Implementation notes](#implementation-notes)
- [Table of Contents](#table-of-contents)
- [Overview](#overview)
- [Deprecation Notice](#deprecation-notice)
- [Message format](#message-format)
- [Flag Values](#flag-values)
- [Protocol](#protocol)
- [Opening a new stream](#opening-a-new-stream)
- [Writing to a stream](#writing-to-a-stream)
- [Closing a stream](#closing-a-stream)
- [Resetting a stream](#resetting-a-stream)
- [Implementation notes](#implementation-notes)

## Overview

Expand All @@ -50,6 +51,41 @@ Implementations in:
- [Go](https://github.com/libp2p/go-mplex)
- [Rust](https://github.com/libp2p/rust-libp2p/tree/master/muxers/mplex)

## Deprecation Notice

**mplex is deprecated** for applications requiring resiliency and liveness.
Users should **prefer QUIC or Yamux**.

**Core limitation: lack of stream flow control**

mplex does not support stream-level flow control, preventing receivers from
applying backpressure. This leads to issues varying from easy to exploit DoS
vulnerabilities due to unbounded sender behavior to hard to debug application
stalls caused by a single slow stream receiver.

**Additional Shortcomings**:
- No stream-level flow control.
- No way for a reader to backpressure a sender.
- No good solution for slow readers.
- Implementations generally do some combination of these mitigations:
- Reset the stream once we reach a certain amount of unread buffered data.
[source](https://github.com/libp2p/rust-libp2p/blob/1c9b3ca355aecffa0bcf83d2495cd4cc1019425b/muxers/mplex/src/config.rs#L118)
- Block operations until the full stream is read from.
[source](https://github.com/libp2p/rust-libp2p/blob/1c9b3ca355aecffa0bcf83d2495cd4cc1019425b/muxers/mplex/src/config.rs#L130)
- Block up to a certain amount of time, then reset the stream.
[source](https://github.com/libp2p/go-mplex/blob/ad9bfb922974b5875cc48c6e7492c4987c0cb94a/multiplex.go#L35-L37)
- Head of line blocking between streams
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as you use TCP, there is no way to resolve this, right? So I think it's a good idea to note that only using QUIC will resolve this and yamux won't.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, you can only really solve head of line blocking of packets with QUIC. However, there's another head of line blocking issue that I'm referencing here, that Yamux can avoid due to stream flow control.

Say you have two streams, and one of them writes a 10 MB payload, and the other one then writes a 1 KB message.

In Yamux, you would send the first 256KB of the first stream, then the next 1KB of the other message. You would not send any more data on the first stream until you receive a window update message.

In mplex, you would try to send the full 10MB payload before you make any progress on the 1KB payload.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just stream level flow control, no? You're mentioning the initial stream flow credits of 256kB.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you rather put this as a sub-bullet under - No stream-level flow control. ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think thats better. Not a strong opinion though.

- For example, No way to interleave data from streams if one stream makes a
big write, and another stream has a small write
- A single slow reader of a single stream can stall the whole application and TCP connection.
- No way of propagating errors.
- Errors that could explain why a peer reset the stream.
- No way to signal to a peer that you will not read any more data (e.g. QUIC's
STOP_SENDING frame).
- Both sides can open streams with the same ID, differing only by who opened the
stream. which may lead to confusion.
- stream names are relatively unused (go-libp2p does not use them. I don't think rust-libp2p or js-libp2p uses them either)

## Message format

Every communication in mplex consists of a header, and a length prefixed data segment.
Expand Down