net: use splice for TCPConn.ReadFrom on Linux

[philhofer]

sendfile only allows the source to be a mmap-able file descriptor. Using splice in the implementation of (net.Conn).ReadFrom allows net.Conn to get the same sort of performance benefits as *os.File when io.Copy is used. (In theory, the src can be any fd.)

Pros:

• Uses the fastest (AFAIK) socket -> socket method available on Linux. (Fundamentally, this is how haproxy works.)
• Can be non-blocking (in the sense that both sockets are talking to the netpoller.)
• Transparent perf improvements to existing users of io.Copy.

Cons:

• Increased implementation complexity: requires two calls to splice, and a call to pipe, along with associated pd.WaitRead()/pd.WaitWrite() business.
• Architecture-specific.

For people writing proxies in Go, this kind of optimization could be huge.

If this sounds agreeable, I can send in a patch next week.

[davecheney]

Can you provide some benchmark numbers to validate this. In the past the
grossness of layering violations have scuttled similar proposals.

On Tue, 26 May 2015 08:51 Philip Hofer notifications@github.com wrote:

sendfile only allows the source to be a mmap-able file descriptor. Using
splice in the implementation of (net.Conn).ReadFrom allows net.Conn to
get the same sort of performance benefits as *os.File when io.Copy is
used. (In theory, the src can be any fd.)

Pros:

• Uses the fastest (AFAIK) socket -> socket method available on Linux.
  (Fundamentally, this is how haproxy works.)
• Can be non-blocking (in the sense that both sockets are talking to
  the netpoller.)
• Transparent perf improvements to existing users of io.Copy.

Cons:

• Increased implementation complexity: requires two calls to splice,
  and a call to pipe, along with associated pd.WaitRead()/pd.WaitWrite()
  business.
• Architecture-specific.

For people writing proxies in Go https://github.com/mailgun/vulcand,
this kind of optimization could be huge.

If this sounds agreeable, I can send in a patch next week.

—
Reply to this email directly or view it on GitHub
#10948.

[minux]

please wait until Go 1.5 is out and Go 1.6 window opens and then we can have a discussion on the mailing list.

[ianlancetaylor]

My impression is that the splice system call won't do the right thing with non-blocking network connections. It will simply return EAGAIN. I'd be happy to hear otherwise.

If we can get better performance by transparently using splice on GNU/Linux, then we should try to do it for 1.6.

[philhofer]

@ianlancetaylor I have a working prototype here that doesn't seem to have that issue.

@minux Ok.

@davecheney Yeah, it's going to take a little while to write an appropriate benchmark. Also, some of the performance is destined to be NIC-dependent.

As an addendum, there is a BSD equivalent (sosplice/somove), although I don't know how it would interact with non-blocking sockets.

[philhofer]

re: benchmarks, here's a simple one:

On a digital ocean single-core VM running Ubuntu 14.04LTS, I tested splicing two TCP connections. In short:

• go io.Copy(dst, src)
• go io.Copy(ioutil.Discard, dst)
• write into src in a loop

benchmark                     old ns/op     new ns/op     delta
BenchmarkSplice1KBchunk       2146          2095          -2.38%
BenchmarkSplice4KBchunk       5603          3753          -33.02%
BenchmarkSplice512KBchunk     579770        362927        -37.40%

benchmark                     old MB/s     new MB/s     speedup
BenchmarkSplice1KBchunk       476.95       488.56       1.02x
BenchmarkSplice4KBchunk       731.01       1091.37      1.49x
BenchmarkSplice512KBchunk     904.30       1444.61      1.60x

Since this is loopback, I'm not sure it's even hitting the NIC, but it's nice to see that the performance is there even without the hardware acceleration.

I'll have access to a 16-core linux machine with a phat NIC next week, so I'll get back to you with numbers more representative of "enterprise-grade" hardware.

[nightlyone]

We could also do better on file2file copies. relevant for the Linux part: http://yarchive.net/comp/linux/splice.html

[philhofer]

@nightlyone The difference ordinary splicing and socket splicing in Go is that sockets are non-blocking. The socket->socket splice is implemented optimally only if it is inside net. A file->file splice can be implemented outside of the standard library without any detriment. (Also a consideration: some filesystems (e.g. FUSE) don't support splice.)

[nightlyone]

@philhofer you are correct, sorry for the noise!

[rfliam]

Just adding a +1 from us here at Comcast for the go1.6 timeframe. We have transfers that run in the 10+MB range as our bread and butter and playing with this for us is around 2x at those sizes.

[huguesb]

FWIW, we rebuilt one of our proxy server with @philhofer 's patch and deployed it in production a few days ago. CPU and memory usage are down. Bandwidth is up. No issues were encountered so far.

Applying patches to the standard library is fairly easy for us as we're doing static builds but it would still be nice to see this feature make into the official tree.