From 63897d69ffeb94f03eb38d73c3d87e8a6f9d6a65 Mon Sep 17 00:00:00 2001 From: David Dias Date: Tue, 28 Mar 2017 15:04:32 +0100 Subject: [PATCH 1/8] first pass --- relay/README.md | 61 +++++++++++++++---------------------------------- 1 file changed, 19 insertions(+), 42 deletions(-) diff --git a/relay/README.md b/relay/README.md index 047d521b7..b33ae78c9 100644 --- a/relay/README.md +++ b/relay/README.md @@ -12,32 +12,18 @@ Table of Contents - [Addressing](#addressing) - [Wire protocol](#wire-protocol) - [Interfaces](#interfaces) -- [Removing existing relay protocol](#removing-existing-relay-protocol) +- [Implementation Details](#implementation-details) ## Overview -The circuit relay is a means of establishing connectivity between -libp2p nodes (such as IPFS) that wouldn't otherwise be able to connect to each other. +The circuit relay is a means of establishing connectivity between libp2p nodes (such as IPFS) that wouldn't otherwise be able to connect to each other. -This helps in situations where nodes are behind NAT or reverse proxies, -or simply don't support the same transports (e.g. go-ipfs vs. browser-ipfs). -libp2p already has modules for NAT ([go-libp2p-nat](https://github.com/libp2p/go-libp2p-nat)), -but these don't always do the job, just because NAT traversal is complicated. -That's why it's useful to have a simple relay protocol. +This helps in situations where nodes are behind NAT or reverse proxies, or simply don't support the same transports (e.g. go-ipfs vs. browser-ipfs). libp2p already has modules for NAT ([go-libp2p-nat](https://github.com/libp2p/go-libp2p-nat)), but these don't always do the job, just because NAT traversal is complicated. That's why it's useful to have a simple relay protocol. -Unlike a transparent **tunnel**, where a libp2p peer would just proxy a -communication stream to a destination (the destination being unaware of the -original source), a circuit-relay makes the destination aware of the original -source and the circuit followed to establish communication between the two. -This provides the destination side with full knowledge of the circuit which, -if needed, could be rebuilt in the opposite direction. +Unlike a transparent **tunnel**, where a libp2p peer would just proxy a communication stream to a destination (the destination being unaware of the original source), a circuit-relay makes the destination aware of the original source and the circuit followed to establish communication between the two. This provides the destination side with full knowledge of the circuit which, if needed, could be rebuilt in the opposite direction. -Apart from that, this relayed connection behaves just like a regular -connection would, but over an existing swarm stream with another peer -(instead of e.g. TCP.): One node asks a relay node to connect to another node -on its behalf. The relay node shortcircuits its streams to the two nodes, -and they are then connected through the relay. +Apart from that, this relayed connection behaves just like a regular connection would, but over an existing swarm stream with another peer (instead of e.g. TCP.): One node asks a relay node to connect to another node on its behalf. The relay node shortcircuits its streams to the two nodes, and they are then connected through the relay. Relayed connections are end-to-end encrypted just like regular connections. @@ -50,18 +36,13 @@ and the swarm protocol is the means to ***relaying*** connections. | QmOne | <------------------------------------> | QmRelay | <-----------------------------------> | QmTwo | +-------+ (/libp2p/relay/circuit multistream) +---------+ (/libp2p/relay/circuit multistream) +-------+ ^ +-----+ ^ - | | | | | /p2p-circuit/QmTwo | | | +--------------------------------------------+ +-------------------------------------------+ ``` -Note: we're using the `/p2p` multiaddr protocol instead of `/ipfs` in this document. -`/ipfs` is currently the canonical way of addressing a libp2p or IPFS node, -but given the growing non-IPFS usage of libp2p, we'll migrate to using `/p2p`. - -Note: at the moment we're not including a mechanism for discovering relay nodes. -For the time being, they should be configured statically. +Note: we're using the `/p2p` multiaddr protocol instead of `/ipfs` in this document. `/ipfs` is currently the canonical way of addressing a libp2p or IPFS node, but given the growing non-IPFS usage of libp2p, we'll migrate to using `/p2p`. +Note: at the moment we're not including a mechanism for discovering relay nodes. For the time being, they should be configured statically. ## Dramatization @@ -131,10 +112,9 @@ TODO: figure out nested relayed connections. ## Wire format -The wire format (or codec) is named `/ipfs/relay/circuit` and is simple. -A variable-length header consisting of two length-prefixed multiaddrs -is followed by a bidirectional stream of arbitrary data, -and the eventual closing of the stream. +The multicodec for the circuit relay protocol is: `/libp2p/relay/circuit`. + +A variable-length header consisting of two length-prefixed multiaddrs is followed by a bidirectional stream of arbitrary data, and the eventual closing of the stream. ``` @@ -149,13 +129,11 @@ and the eventual closing of the stream. +------------ multiaddr of the dialing node ``` -After getting a stream to the relay node from its libp2p swarm, -the dialing transport writes the header to the stream. -The relaying node reads the header, gets a stream to the destination node, -then writes the header to the destination stream and shortcircuits the two streams. +After getting a stream to the relay node from its libp2p swarm, the dialing transport writes the header to the stream. -Each relayed connection corresponds to two multistreams, -one between QmOne and QmRelay, the other between QmRelay and QmTwo. +The relaying node reads the header, gets a stream to the destination node, then writes the header to the destination stream and shortcircuits the two streams. + +Each relayed connection corresponds to two multistreams, one between QmOne and QmRelay, the other between QmRelay and QmTwo. Implementation details: - The relay node has the `Swarm.EnableRelaying` config option enabled @@ -166,9 +144,7 @@ Implementation details: ## Interfaces -As explained above, the relay is both a transport (`tpt.Transport`) -and a mounted stream protocol (`p2pnet.StreamHandler`). -In addition it provides a means of specifying relay nodes to listen/dial through. +As explained above, the relay is both a transport (`tpt.Transport`) and a mounted stream protocol (`p2pnet.StreamHandler`). In addition it provides a means of specifying relay nodes to listen/dial through. TODO: the usage of p2pnet.StreamHandler is a little bit off, but it gets the point across. @@ -193,7 +169,9 @@ fund NewCircuitRelay(h p2phost.Host) ``` -### Removing existing relay protocol +## Implementation details + +### Removing existing relay protocol in Go Note that there is an existing swarm protocol colloqiually called relay. It lives in the go-libp2p package and is named `/ipfs/relay/line/0.1.0`. @@ -208,5 +186,4 @@ It lives in the go-libp2p package and is named `/ipfs/relay/line/0.1.0`. - Capable of *accepting* connections, and *relaying* connections. - Not capable of *connecting* via relaying. -Since the existing protocol is incomplete, insecure, and certainly not used, -we can safely remove it. +Since the existing protocol is incomplete, insecure, and certainly not used, we can safely remove it. From 4c5a51c980152ad126ff1b67ce2fe4ea969522bd Mon Sep 17 00:00:00 2001 From: David Dias Date: Tue, 28 Mar 2017 15:53:56 +0100 Subject: [PATCH 2/8] update addresses examples --- relay/README.md | 61 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 40 insertions(+), 21 deletions(-) diff --git a/relay/README.md b/relay/README.md index b33ae78c9..762ddbafd 100644 --- a/relay/README.md +++ b/relay/README.md @@ -68,47 +68,66 @@ Scene 2: ## Addressing -`/p2p-circuit` multiaddrs don't carry any meaning of their own. -They need to encapsulate a `/p2p` address, or -be encapsulated in a `/p2p` address, or both. +`/p2p-circuit` multiaddrs don't carry any meaning of their own. They need to encapsulate a `/p2p` address, or be encapsulated in a `/p2p` address, or both. -As with all other multiaddrs, encapsulation of different protocols -determines which metaphorical tubes to connect to each other. +As with all other multiaddrs, encapsulation of different protocols determines which metaphorical tubes to connect to each other. + +A `/p2p-circuit` circuit address, is formated following: + +`/p2p-circuit[]` + +This opens the room for multiple hop relay, where the first relay is encapsulated in the seconf relay multiaddr, such as: + +`/p2p-circuit/p2p-circuit///` A few examples: +Using any relay available: + - `/p2p-circuit/p2p/QmTwo` - - Dial QmTwo, through any available relay node. - - The relay node will use peer routing to find an address for QmTwo. -- `/p2p/QmRelay/p2p-circuit/p2p/QmTwo` - - Dial QmTwo, through QmRelay. - - Use peer routing to find an address for QmRelay. - - The relay node will also use peer routing, to find an address for QmTwo. + - Dial QmTwo, through any available relay node (or find one node that can relay). + - The relay node will use peer routing to find an address for QmTwo if it doesn't have a direct connection. - `/p2p-circuit/ip4/../tcp/../p2p/QmTwo` - Dial QmTwo, through any available relay node, but force the relay node to use the encapsulated `/ip4` multiaddr for connecting to QmTwo. - We'll probably not support forced addresses for now, just because it's complicated. -- `/ip4/../tcp/../p2p/QmRelay/p2p-circuit` - - Listen for connections relayed through QmRelay. + +Specify a relay: + +- `/p2p-circuit/p2p/QmRelay/p2p/QmTwo` + - Dial QmTwo, through QmRelay. + - Use peer routing to find an address for QmRelay. + - The relay node will also use peer routing, to find an address for QmTwo. +- `/p2p-circuit/ip4/../tcp/../p2p/QmRelay/p2p/QmTwo` + - Dial QmTwo, through QmRelay. - Includes info for connecting to QmRelay. - - Also makes QmRelay available for relayed dialing, based on how listeners currently relate to dialers. -- `/p2p/QmRelay/p2p-circuit` - - Same as previous example, but use peer routing to find an address for QmRelay. + - The relay node will use peer routing to find an address for QmTwo. + +Double relay: + - `/p2p-circuit/p2p/QmTwo/p2p-circuit/p2p/QmThree` - Dial QmThree, through a relayed connection to QmTwo. - The relay nodes will use peer routing to find an address for QmTwo and QmThree. - We'll probably not support nested relayed connections for now, there are edge cases to think of. -- `/ip4/../tcp/../p2p/QmRelay/p2p-circuit/p2p/QmTwo` - - Dial QmTwo, through QmRelay. + +-- + +?? I don't understand the usage of the following: + +- `/ip4/../tcp/../p2p/QmRelay/p2p-circuit` + - Listen for connections relayed through QmRelay. - Includes info for connecting to QmRelay. - - The relay node will use peer routing to find an address for QmTwo. + - Also makes QmRelay available for relayed dialing, based on how listeners currently relate to dialers. +- `/p2p/QmRelay/p2p-circuit` + - Same as previous example, but use peer routing to find an address for QmRelay. + +?? I believe we don't need this one: - `/p2p-circuit` - Use relay discovery to find a suitable relay node. (Neither specified nor implemented.) - Listen for relayed connections. - Dial through the discovered relay node for any `/p2p-circuit` multiaddr. -TODO: figure out forced addresses. -TODO: figure out nested relayed connections. +TODO: figure out forced addresses. -> what is forced addresses? ## Wire format From b74a39e9fc9e82f78827196403b04983ecf3d6b3 Mon Sep 17 00:00:00 2001 From: David Dias Date: Tue, 28 Mar 2017 16:44:32 +0100 Subject: [PATCH 3/8] add error table --- relay/README.md | 51 +++++++++++++++++++++++++++++++++---------------- 1 file changed, 35 insertions(+), 16 deletions(-) diff --git a/relay/README.md b/relay/README.md index 762ddbafd..e78a51e7e 100644 --- a/relay/README.md +++ b/relay/README.md @@ -2,11 +2,13 @@ > Circuit Switching in libp2p -Implementations -- [js-libp2p-circuit](https://github.com/libp2p/js-libp2p-circuit) -- status: started -- [go-libp2p-circuit](https://github.com/libp2p/go-libp2p-circuit) -- status: ready-to-start +## Implementations + +- [js-libp2p-circuit](https://github.com/libp2p/js-libp2p-circuit) +- [go-libp2p-circuit](https://github.com/libp2p/go-libp2p-circuit) + +## Table of Contents -Table of Contents - [Overview](#overview) - [Dramatization](#dramatization) - [Addressing](#addressing) @@ -14,7 +16,6 @@ Table of Contents - [Interfaces](#interfaces) - [Implementation Details](#implementation-details) - ## Overview The circuit relay is a means of establishing connectivity between libp2p nodes (such as IPFS) that wouldn't otherwise be able to connect to each other. @@ -28,16 +29,15 @@ Apart from that, this relayed connection behaves just like a regular connection Relayed connections are end-to-end encrypted just like regular connections. The circuit relay is both a tunneled transport and a mounted swarm protocol. -The transport is the means of ***establishing*** and ***accepting*** connections, -and the swarm protocol is the means to ***relaying*** connections. +The transport is the means of ***establishing*** and ***accepting*** connections, and the swarm protocol is the means to ***relaying*** connections. ``` -+-------+ /ip4/.../tcp/.../ws/p2p/QmRelay +---------+ /ip4/.../tcp/.../p2p/QmTwo +-------+ -| QmOne | <------------------------------------> | QmRelay | <-----------------------------------> | QmTwo | -+-------+ (/libp2p/relay/circuit multistream) +---------+ (/libp2p/relay/circuit multistream) +-------+ - ^ +-----+ ^ - | /p2p-circuit/QmTwo | | | - +--------------------------------------------+ +-------------------------------------------+ ++-----+ /ip4/.../tcp/.../ws/p2p/QmRelay +-------+ /ip4/.../tcp/.../p2p/QmTwo +-----+ +|QmOne| <------------------------------------>|QmRelay|<----------------------------------->|QmTwo| ++-----+ (/libp2p/relay/circuit multistream) +-------+ (/libp2p/relay/circuit multistream) +-----+ + ^ +-----+ ^ + | /p2p-circuit/QmTwo | | | + +-----------------------------------------+ +-----------------------------------------+ ``` Note: we're using the `/p2p` multiaddr protocol instead of `/ipfs` in this document. `/ipfs` is currently the canonical way of addressing a libp2p or IPFS node, but given the growing non-IPFS usage of libp2p, we'll migrate to using `/p2p`. @@ -65,7 +65,6 @@ Scene 2: - QmOne tries to connect via relaying, because it shares this transport with QmTwo. - A lively and prolonged dialogue ensues. - ## Addressing `/p2p-circuit` multiaddrs don't carry any meaning of their own. They need to encapsulate a `/p2p` address, or be encapsulated in a `/p2p` address, or both. @@ -131,13 +130,14 @@ TODO: figure out forced addresses. -> what is forced addresses? ## Wire format +> out of date + The multicodec for the circuit relay protocol is: `/libp2p/relay/circuit`. A variable-length header consisting of two length-prefixed multiaddrs is followed by a bidirectional stream of arbitrary data, and the eventual closing of the stream. ``` - ^ ^ ^ | | | | | +-- bidirectional data stream @@ -160,6 +160,26 @@ Implementation details: - The relay node validates the `src` header field. - The listening node validates the `dst` header field. +### Error table + +This is a table of error codes and sample messages that may occur during a relay setup. Codes in the 200 range are returned by the relay node. Codes in the 300 range are returned by the destination node. + + +| Code | Message | Meaning | +| ----- |:-----------------:|:----------:| +| 100 | OK | Relay was setup correctly | +| 220 | "src address too long" | | +| 221 | "dst address too long" | | +| 250 | "failed to parse src addr: no such protocol ipfs" | The `` multiaddr in the header was invalid | +| 251 | "failed to parse dst addr: no such protocol ipfs" | The `` multiaddr in the header was invalid | +| 260 | "passive relay has no connection to dst" | | +| 261 | "active relay could not connect to dst: connection refused" | relay could not form new connection to target peer | +| 262 | "could not open new stream to dst: BAD ERROR" | relay has connection to dst, but failed to open a new stream | +| 270 | " does not support relay" | | +| 320 | "src address too long" | | +| 321 | "dst address too long" | | +| 350 | "failed to parse src addr: no such protocol ifps" | The `` multiaddr in the header was invalid | +| 351 | "failed to parse dst addr: no such protocol ifps" | The `` multiaddr in the header was invalid | ## Interfaces @@ -187,7 +207,6 @@ type CircuitRelay interface { fund NewCircuitRelay(h p2phost.Host) ``` - ## Implementation details ### Removing existing relay protocol in Go From 5ad252dfcdf04c42be38fb4fb62bbbaa00ea074f Mon Sep 17 00:00:00 2001 From: David Dias Date: Tue, 28 Mar 2017 17:18:49 +0100 Subject: [PATCH 4/8] update wire format section with why notes --- relay/README.md | 104 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 84 insertions(+), 20 deletions(-) diff --git a/relay/README.md b/relay/README.md index e78a51e7e..f54ffe296 100644 --- a/relay/README.md +++ b/relay/README.md @@ -130,35 +130,99 @@ TODO: figure out forced addresses. -> what is forced addresses? ## Wire format -> out of date +### Overview -The multicodec for the circuit relay protocol is: `/libp2p/relay/circuit`. +#### Setup -A variable-length header consisting of two length-prefixed multiaddrs is followed by a bidirectional stream of arbitrary data, and the eventual closing of the stream. +Peers involved: +- A, B, R +- A wants to connect to B, but needs to relay through R +#### Assumptions + +A has connection to R, R has connection to B + +#### Process + +- A opens new stream `sAR` to R using protocol RELAY +- A writes Bs multiaddr `/ipfs/QmB` on `sAR` +- R receives stream `sAR` and reads `/ipfs/QmB` from it. +- R opens a new stream `sRB` to B using protocol RELAY +- R writes `/ipfs/QmB` on `sRB` +- B receives stream `sRB` and reads `/ipfs/QmB` from it. +- B sees that the multiaddr it read is its own and chooses to handle this stream as an endpoint instead of attempting to relay further +- TODO: step for R to send back status code to A +- R now pipes `sAR` and `sRB` together +- TODO: step for B to send back status code to A +- B passes stream to `NewConnHandler` to be handled like any other new incoming connection + +### Under the microscope + +Peer A wants to connect to peer B through peer R. + +`maddrA` is peer A's multiaddr +`maddrR` is peer R's multiaddr +`maddrB` is peer B's multiaddr +`maxAddrLen` is arbitrarily 1024 + +#### Function definitions +##### Process for reading a multiaddr +We define `readLpMaddr` to be the following: + +Read a Uvarint `V` from the stream. If the value is higher +than `maxAddrLen`, (write an error message back?) close the +stream and halt the relay attempt. + +Then read `V` bytes from the stream and checks if its a valid multiaddr. +If it is not a valid multiaddr (write an error back?) close the stream and return. + +#### Opening a relay +Peer A opens a new stream to R on the 'hop' protocol and writes: ``` - - ^ ^ ^ - | | | - | | +-- bidirectional data stream - | | (usually /multistream-select in the case of /p2p multiaddrs) - | | - | +------- multiaddr of the listening node - | - +------------ multiaddr of the dialing node + ``` -After getting a stream to the relay node from its libp2p swarm, the dialing transport writes the header to the stream. +It then waits for a response in the form of: +``` + +``` -The relaying node reads the header, gets a stream to the destination node, then writes the header to the destination stream and shortcircuits the two streams. +Once it receives that, it checks if the status code is `OK`. If it is, it passes the new connection to its `NewConnHandler`. +Otherwise, it returns the error message to the caller. -Each relayed connection corresponds to two multistreams, one between QmOne and QmRelay, the other between QmRelay and QmTwo. +### 'hop' protocol handler + +Peer R receives a new stream on the 'hop' protocol. +It then calls `readLpMaddr` twice. The first value is `` and the second is ``. +Peer R checks to make sure that `` matches the remote peer of the stream its reading +from. If it does not match, it (writes an error back?) closes the stream and halts the relay attempt. + +Peer R checks if `` refers to itself, if it does, it (writes an error back?) closes the stream and halts the relay attempt. +Peer R then checks if it has an open connection to the peer specified by ``. +If it does not, and the relay is not an "active" relay it (writes an error back) closes the stream, and halts the relay attempt. +If R does not have a connection to ``, and it *is* an "active" relay, it attempts to connect to ``. +If this connection succeeds it continues, otherwise it (writes back an error) closes the stream, and halts the relay attempt. +R now opens a new stream to B with the 'stop' relay protocol ID, and writes: +``` + +``` + +After this, R simply pipes the stream from A and the stream it opened to B together. R's job is complete. + +### 'stop' protocol handler + +Peer B receives a new stream on the 'stop' protocol. It then calls `readLpMaddr` twice on this stream. +The first value is `` and the second value is ``. Any error from those calls should be written back accordingly. + +B now verifies that `` matches its peer ID. It then also checks that `` is valid. It uses src as the +'remote addr' of the new 'incoming relay connection' it will create. + +Peer B now writes back a message of the form: +``` + +``` -Implementation details: -- The relay node has the `Swarm.EnableRelaying` config option enabled -- The relay node allows only one relayed connection between any two nodes. -- The relay node validates the `src` header field. -- The listening node validates the `dst` header field. +And passes the relayed connection into its `NewConnHandler`. ### Error table From d65d541f5e601e5eb444c3a4e994101b185114b6 Mon Sep 17 00:00:00 2001 From: Dmitriy Ryajov Date: Thu, 30 Mar 2017 00:59:53 -0700 Subject: [PATCH 5/8] Feat/update relay spec (#16) * feat: adding err code 280 "can't relay to itlsef" * doc: reworded 280 description --- relay/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/relay/README.md b/relay/README.md index f54ffe296..08d8bb17a 100644 --- a/relay/README.md +++ b/relay/README.md @@ -240,6 +240,7 @@ This is a table of error codes and sample messages that may occur during a relay | 261 | "active relay could not connect to dst: connection refused" | relay could not form new connection to target peer | | 262 | "could not open new stream to dst: BAD ERROR" | relay has connection to dst, but failed to open a new stream | | 270 | " does not support relay" | | +| 280 | "can't relay to itself" | The relay got its own address as destination| | 320 | "src address too long" | | | 321 | "dst address too long" | | | 350 | "failed to parse src addr: no such protocol ifps" | The `` multiaddr in the header was invalid | From 5105b41f0531821d3fb53d8c7b0b5f1a2f800094 Mon Sep 17 00:00:00 2001 From: David Dias Date: Mon, 8 May 2017 20:19:30 +0200 Subject: [PATCH 6/8] reviewed Overview, Dramatization and Addressing --- relay/README.md | 63 ++++++++++++++++++++----------------------------- 1 file changed, 26 insertions(+), 37 deletions(-) diff --git a/relay/README.md b/relay/README.md index 08d8bb17a..019d03d64 100644 --- a/relay/README.md +++ b/relay/README.md @@ -1,6 +1,6 @@ # Circuit Relay -> Circuit Switching in libp2p +> Circuit Switching for libp2p, also known as TURN or Relay in Networking literature. ## Implementations @@ -18,9 +18,9 @@ ## Overview -The circuit relay is a means of establishing connectivity between libp2p nodes (such as IPFS) that wouldn't otherwise be able to connect to each other. +The circuit relay is a means of establishing connectivity between libp2p nodes (such as IPFS) that wouldn't otherwise be able to establish a direct connection to each other. -This helps in situations where nodes are behind NAT or reverse proxies, or simply don't support the same transports (e.g. go-ipfs vs. browser-ipfs). libp2p already has modules for NAT ([go-libp2p-nat](https://github.com/libp2p/go-libp2p-nat)), but these don't always do the job, just because NAT traversal is complicated. That's why it's useful to have a simple relay protocol. +This helps in situations where nodes are behind NAT or reverse proxies, or simply don't support the same transports (e.g. go-ipfs vs. browser-ipfs). libp2p already has modules for NAT ([go-libp2p-nat](https://github.com/libp2p/go-libp2p-nat)), however piercing through NATs is not always a option due to their implementation differences. The circuit relay protocol exist to overcome those scenarios. Unlike a transparent **tunnel**, where a libp2p peer would just proxy a communication stream to a destination (the destination being unaware of the original source), a circuit-relay makes the destination aware of the original source and the circuit followed to establish communication between the two. This provides the destination side with full knowledge of the circuit which, if needed, could be rebuilt in the opposite direction. @@ -40,9 +40,9 @@ The transport is the means of ***establishing*** and ***accepting*** connections +-----------------------------------------+ +-----------------------------------------+ ``` -Note: we're using the `/p2p` multiaddr protocol instead of `/ipfs` in this document. `/ipfs` is currently the canonical way of addressing a libp2p or IPFS node, but given the growing non-IPFS usage of libp2p, we'll migrate to using `/p2p`. +## Notes for the reader -Note: at the moment we're not including a mechanism for discovering relay nodes. For the time being, they should be configured statically. +1. We're using the `/p2p` multiaddr protocol instead of `/ipfs` in this document. `/ipfs` is currently the canonical way of addressing a libp2p or IPFS node, but given the growing non-IPFS usage of libp2p, we'll migrate to using `/p2p`. ## Dramatization @@ -73,11 +73,16 @@ As with all other multiaddrs, encapsulation of different protocols determines wh A `/p2p-circuit` circuit address, is formated following: -`/p2p-circuit[]` +`[]/p2p-circuit/` -This opens the room for multiple hop relay, where the first relay is encapsulated in the seconf relay multiaddr, such as: +Examples: -`/p2p-circuit/p2p-circuit///` +- `/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - No known relay +- `/ip4/127.0.0.1/tcp/5002/ipfs/QmdPU7PfRyKehdrP5A3WqmjyD6bhVpU1mLGKppa2FjGDjZ/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - Known relay + +This opens the room for multiple hop relay, where the first relay is encapsulated in the second relay multiaddr, such as: + +`<1st relay>/p2p-circuit/<2nd relay>/p2p-circuit/` A few examples: @@ -87,17 +92,15 @@ Using any relay available: - Dial QmTwo, through any available relay node (or find one node that can relay). - The relay node will use peer routing to find an address for QmTwo if it doesn't have a direct connection. - `/p2p-circuit/ip4/../tcp/../p2p/QmTwo` - - Dial QmTwo, through any available relay node, - but force the relay node to use the encapsulated `/ip4` multiaddr for connecting to QmTwo. - - We'll probably not support forced addresses for now, just because it's complicated. + - Dial QmTwo, through any available relay node, but force the relay node to use the encapsulated `/ip4` multiaddr for connecting to QmTwo. Specify a relay: -- `/p2p-circuit/p2p/QmRelay/p2p/QmTwo` +- `/p2p/QmRelay/p2p-circuit/p2p/QmTwo` - Dial QmTwo, through QmRelay. - Use peer routing to find an address for QmRelay. - The relay node will also use peer routing, to find an address for QmTwo. -- `/p2p-circuit/ip4/../tcp/../p2p/QmRelay/p2p/QmTwo` +- `/ip4/../tcp/../p2p/QmRelay/p2p-circuit/p2p/QmTwo` - Dial QmTwo, through QmRelay. - Includes info for connecting to QmRelay. - The relay node will use peer routing to find an address for QmTwo. @@ -109,25 +112,6 @@ Double relay: - The relay nodes will use peer routing to find an address for QmTwo and QmThree. - We'll probably not support nested relayed connections for now, there are edge cases to think of. --- - -?? I don't understand the usage of the following: - -- `/ip4/../tcp/../p2p/QmRelay/p2p-circuit` - - Listen for connections relayed through QmRelay. - - Includes info for connecting to QmRelay. - - Also makes QmRelay available for relayed dialing, based on how listeners currently relate to dialers. -- `/p2p/QmRelay/p2p-circuit` - - Same as previous example, but use peer routing to find an address for QmRelay. - -?? I believe we don't need this one: -- `/p2p-circuit` - - Use relay discovery to find a suitable relay node. (Neither specified nor implemented.) - - Listen for relayed connections. - - Dial through the discovered relay node for any `/p2p-circuit` multiaddr. - -TODO: figure out forced addresses. -> what is forced addresses? - ## Wire format ### Overview @@ -142,14 +126,14 @@ Peers involved: A has connection to R, R has connection to B -#### Process +#### Sequence of events - A opens new stream `sAR` to R using protocol RELAY -- A writes Bs multiaddr `/ipfs/QmB` on `sAR` -- R receives stream `sAR` and reads `/ipfs/QmB` from it. +- A writes Bs multiaddr `/p2p/QmB` on `sAR` +- R receives stream `sAR` and reads `/p2p/QmB` from it. - R opens a new stream `sRB` to B using protocol RELAY -- R writes `/ipfs/QmB` on `sRB` -- B receives stream `sRB` and reads `/ipfs/QmB` from it. +- R writes `/p2p/QmB` on `sRB` +- B receives stream `sRB` and reads `/p2p/QmB` from it. - B sees that the multiaddr it read is its own and chooses to handle this stream as an endpoint instead of attempting to relay further - TODO: step for R to send back status code to A - R now pipes `sAR` and `sRB` together @@ -290,3 +274,8 @@ It lives in the go-libp2p package and is named `/ipfs/relay/line/0.1.0`. - Not capable of *connecting* via relaying. Since the existing protocol is incomplete, insecure, and certainly not used, we can safely remove it. + +## Future work + +- Multihop relay - With this specification, we are only enabling single hop relays to exist. Multihop relay will come at a later stage as Packet Switching. +- Relay discovery mechanism - At the moment we're not including a mechanism for discovering relay nodes. For the time being, they should be configured statically. From 9ca982bf820927204aadec090a39b9e1ce12201d Mon Sep 17 00:00:00 2001 From: David Dias Date: Tue, 9 May 2017 19:34:43 +0200 Subject: [PATCH 7/8] apply lars CR --- relay/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/relay/README.md b/relay/README.md index 019d03d64..e43102c57 100644 --- a/relay/README.md +++ b/relay/README.md @@ -20,7 +20,7 @@ The circuit relay is a means of establishing connectivity between libp2p nodes (such as IPFS) that wouldn't otherwise be able to establish a direct connection to each other. -This helps in situations where nodes are behind NAT or reverse proxies, or simply don't support the same transports (e.g. go-ipfs vs. browser-ipfs). libp2p already has modules for NAT ([go-libp2p-nat](https://github.com/libp2p/go-libp2p-nat)), however piercing through NATs is not always a option due to their implementation differences. The circuit relay protocol exist to overcome those scenarios. +This helps in situations where nodes are behind NAT or reverse proxies, or simply don't support the same transports (e.g. go-ipfs vs. browser-ipfs). libp2p already has modules for NAT ([go-libp2p-nat](https://github.com/libp2p/go-libp2p-nat)), however piercing through NATs is not always an option due to their implementation differences. The circuit relay protocol exists to overcome those scenarios. Unlike a transparent **tunnel**, where a libp2p peer would just proxy a communication stream to a destination (the destination being unaware of the original source), a circuit-relay makes the destination aware of the original source and the circuit followed to establish communication between the two. This provides the destination side with full knowledge of the circuit which, if needed, could be rebuilt in the opposite direction. @@ -77,10 +77,10 @@ A `/p2p-circuit` circuit address, is formated following: Examples: -- `/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - No known relay -- `/ip4/127.0.0.1/tcp/5002/ipfs/QmdPU7PfRyKehdrP5A3WqmjyD6bhVpU1mLGKppa2FjGDjZ/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - Known relay +- `/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - Arbitrary relay node +- `/ip4/127.0.0.1/tcp/5002/ipfs/QmdPU7PfRyKehdrP5A3WqmjyD6bhVpU1mLGKppa2FjGDjZ/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - Specific relay node -This opens the room for multiple hop relay, where the first relay is encapsulated in the second relay multiaddr, such as: +This opens the room for multiple hop relay, where the second relay is encapsulated in the first relay multiaddr, such as: `<1st relay>/p2p-circuit/<2nd relay>/p2p-circuit/` From 1759516ead48006d0088e8bbe00f3bdf4240bf22 Mon Sep 17 00:00:00 2001 From: David Dias Date: Tue, 9 May 2017 20:38:39 +0200 Subject: [PATCH 8/8] review Wire protocol section from the relay spec --- relay/README.md | 119 ++++++++++++++++++++++++------------------------ 1 file changed, 59 insertions(+), 60 deletions(-) diff --git a/relay/README.md b/relay/README.md index e43102c57..e2d8ddc7f 100644 --- a/relay/README.md +++ b/relay/README.md @@ -114,54 +114,46 @@ Double relay: ## Wire format -### Overview +We start the description of the Wire format by illustrating a possible flow scenario and then describing them in detail by phases. -#### Setup +### High level overview of establishing a relayed connection -Peers involved: -- A, B, R +Setup: +- Peers involved, A, B, R - A wants to connect to B, but needs to relay through R -#### Assumptions - -A has connection to R, R has connection to B - -#### Sequence of events - -- A opens new stream `sAR` to R using protocol RELAY -- A writes Bs multiaddr `/p2p/QmB` on `sAR` -- R receives stream `sAR` and reads `/p2p/QmB` from it. -- R opens a new stream `sRB` to B using protocol RELAY -- R writes `/p2p/QmB` on `sRB` -- B receives stream `sRB` and reads `/p2p/QmB` from it. -- B sees that the multiaddr it read is its own and chooses to handle this stream as an endpoint instead of attempting to relay further -- TODO: step for R to send back status code to A -- R now pipes `sAR` and `sRB` together -- TODO: step for B to send back status code to A -- B passes stream to `NewConnHandler` to be handled like any other new incoming connection +Assumptions: +- A has connection to R, R has connection to B + +Events: +- phase I: Open a request for a relayed stream (A to R) + - A dials a new stream `sAR` to R using protocol `/libp2p/relay/hop` + - A writes Bs multiaddr `/p2p/QmB` on `sAR` + - R receives stream `sAR` and reads `/p2p/QmB` from it. +- phase II: Open a stream to be relayed (R to B) + - R opens a new stream `sRB` to B using protocol `/libp2p/relay/stop` + - R writes `/p2p/QmA` on `sRB` + - R writes OK to `sAR` +- phase III: Streams are piped together, establishing a circuit + - B receives stream `sRB` and reads `/p2p/QmA` from it. + - B writes OK to `sRB` + - B passes stream to `NewConnHandler` to be handled like any other new incoming connection ### Under the microscope -Peer A wants to connect to peer B through peer R. - -`maddrA` is peer A's multiaddr -`maddrR` is peer R's multiaddr -`maddrB` is peer B's multiaddr -`maxAddrLen` is arbitrarily 1024 +Notes for the reader: +- `maddrA` is peer A's multiaddr +- `maddrR` is peer R's multiaddr +- `maddrB` is peer B's multiaddr +- `maxAddrLen` is arbitrarily 1024 +- `readLpMaddr` is a function that does the following: -#### Function definitions -##### Process for reading a multiaddr -We define `readLpMaddr` to be the following: +Read a Uvarint `V` from the stream. If the value is higher than `maxAddrLen`, writes an error code back, closes the stream and halts the relay attempt. If not, then reads `V` bytes from the stream and checks if its a valid multiaddr. Check for multiaddr validity and error back if not a valid formated multiaddr. -Read a Uvarint `V` from the stream. If the value is higher -than `maxAddrLen`, (write an error message back?) close the -stream and halt the relay attempt. +#### Phase I -Then read `V` bytes from the stream and checks if its a valid multiaddr. -If it is not a valid multiaddr (write an error back?) close the stream and return. +Peer A opens a new stream to R on the '/libp2p/relay/hop' protocol and writes: -#### Opening a relay -Peer A opens a new stream to R on the 'hop' protocol and writes: ``` ``` @@ -171,31 +163,38 @@ It then waits for a response in the form of: ``` -Once it receives that, it checks if the status code is `OK`. If it is, it passes the new connection to its `NewConnHandler`. -Otherwise, it returns the error message to the caller. +Once it receives that, it checks if the status code is `OK`. If it is, it passes the new connection to its `NewConnHandler`. Otherwise, it returns the error message to the caller. -### 'hop' protocol handler +Peer R receives a new stream on the '/libp2p/relay/hop' protocol. -Peer R receives a new stream on the 'hop' protocol. It then calls `readLpMaddr` twice. The first value is `` and the second is ``. -Peer R checks to make sure that `` matches the remote peer of the stream its reading -from. If it does not match, it (writes an error back?) closes the stream and halts the relay attempt. + +Peer R checks to make sure that `` matches the remote peer of the stream its reading from. If it does not match, it writes an Error message, closes the stream and halts the relay attempt. Peer R checks if `` refers to itself, if it does, it (writes an error back?) closes the stream and halts the relay attempt. + Peer R then checks if it has an open connection to the peer specified by ``. If it does not, and the relay is not an "active" relay it (writes an error back) closes the stream, and halts the relay attempt. + If R does not have a connection to ``, and it *is* an "active" relay, it attempts to connect to ``. + If this connection succeeds it continues, otherwise it (writes back an error) closes the stream, and halts the relay attempt. -R now opens a new stream to B with the 'stop' relay protocol ID, and writes: + + +#### Phase II + +R now opens a new stream to B with the '/libp2p/relay/stop' relay protocol multicodec and writes: + ``` ``` -After this, R simply pipes the stream from A and the stream it opened to B together. R's job is complete. +After this, R pipes the stream from A and the stream it opened to B together. R also writes OK back to A. R's job is complete. + +#### Phase III -### 'stop' protocol handler +Peer B receives a new stream on the '/libp2p/relay/stop' protocol. It then calls `readLpMaddr` twice on this stream. -Peer B receives a new stream on the 'stop' protocol. It then calls `readLpMaddr` twice on this stream. The first value is `` and the second value is ``. Any error from those calls should be written back accordingly. B now verifies that `` matches its peer ID. It then also checks that `` is valid. It uses src as the @@ -208,25 +207,25 @@ Peer B now writes back a message of the form: And passes the relayed connection into its `NewConnHandler`. -### Error table +### Status codes table -This is a table of error codes and sample messages that may occur during a relay setup. Codes in the 200 range are returned by the relay node. Codes in the 300 range are returned by the destination node. +This is a table of status codes and sample messages that may occur during a relay setup. Codes in the 200 range are returned by the relay node. Codes in the 300 range are returned by the destination node. -| Code | Message | Meaning | -| ----- |:-----------------:|:----------:| -| 100 | OK | Relay was setup correctly | -| 220 | "src address too long" | | -| 221 | "dst address too long" | | +| Code | Message | Meaning | +| ----- |:--------------------------------------------------|:----------:| +| 100 | OK | Relay was setup correctly | +| 220 | "src address too long" | | +| 221 | "dst address too long" | | | 250 | "failed to parse src addr: no such protocol ipfs" | The `` multiaddr in the header was invalid | | 251 | "failed to parse dst addr: no such protocol ipfs" | The `` multiaddr in the header was invalid | -| 260 | "passive relay has no connection to dst" | | +| 260 | "passive relay has no connection to dst" | | | 261 | "active relay could not connect to dst: connection refused" | relay could not form new connection to target peer | -| 262 | "could not open new stream to dst: BAD ERROR" | relay has connection to dst, but failed to open a new stream | -| 270 | " does not support relay" | | -| 280 | "can't relay to itself" | The relay got its own address as destination| -| 320 | "src address too long" | | -| 321 | "dst address too long" | | +| 262 | "could not open new stream to dst: BAD ERROR" | relay has connection to dst, but failed to open a new stream | +| 270 | " does not support relay" | | +| 280 | "can't relay to itself" | The relay got its own address as destination | +| 320 | "src address too long" | | +| 321 | "dst address too long" | | | 350 | "failed to parse src addr: no such protocol ifps" | The `` multiaddr in the header was invalid | | 351 | "failed to parse dst addr: no such protocol ifps" | The `` multiaddr in the header was invalid |