net/http/httputil: ReverseProxy, CloseNotify and HTTP 1.1 pipelining

[juliens]

What version of Go are you using (go version)?

go version go1.10 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/juliens/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/juliens/dev/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build191630190=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I use httputil.ReverseProxy in order to proxify an apt mirror.

What did you expect to see?

My reverse proxy works when I try to do apt update/install

What did you see instead?

apt receive:

502 Bad Gateway

and in reverse proxy logs:

2018/02/16 13:52:30 http: proxy error: context canceled

More

To reproduce, you can do this reverse proxying (on a linux distrib with apt)

package main

import (
   "net/http/httputil"
   "net/url"
   "net/http"
)

func main() {
   backend, _ := url.Parse("http://archive.ubuntu.com")
   proxy := httputil.NewSingleHostReverseProxy(backend)

   handler := http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
      proxy.ServeHTTP(rw, req)
   })

   http.ListenAndServe(":80", handler)
}

Then modify in your /etc/hosts

127.0.0.1   archive.ubuntu.com

and try

apt update

or

apt install ...

[namusyaka]

/cc @bradfitz @tombergan

[fraenkel]

Works for me using ubuntu 17.10 and Go 1.10

[gopherbot]

Change https://golang.org/cl/123875 mentions this issue: net/http: don't cancel Request.Context on pipelined Server requests