Dockerfile: bump up components (Kubernetes 1.20, ...) #197

AkihiroSuda · 2020-12-16T05:54:38Z

~~- Go: 1.13 -> 1.15 (incurs s/go install/go get/g)~~

containerd: 1.4.2 -> 1.4.3
CNI plugins: 0.8.6 -> 0.9.0
kind node: 1.19.0 -> 1.20.0

AkihiroSuda · 2020-12-16T07:26:18Z

?   	github.com/containerd/stargz-snapshotter/script/benchmark/hello-bench/src/go	[no test files]
fatal error: checkptr: converted pointer straddles multiple allocations

goroutine 24 [running]:
runtime.throw(0xa3a0c2, 0x3a)
	/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc0000b2778 sp=0xc0000b2748 pc=0x469992
runtime.checkptrAlignment(0xc0000d7150, 0x9f7140, 0x1)
	/usr/local/go/src/runtime/checkptr.go:20 +0xc9 fp=0xc0000b27a8 sp=0xc0000b2778 pc=0x438949
go.etcd.io/bbolt.(*Bucket).write(0xc0000b2918, 0x0, 0x0, 0x0)
	/go/pkg/mod/go.etcd.io/[email protected]/bucket.go:624 +0x15f fp=0xc0000b2810 sp=0xc0000b27a8 pc=0x7f669f
go.etcd.io/bbolt.(*Bucket).CreateBucket(0xc0001a60f8, 0xcea348, 0x2, 0x2, 0xc0001a4110, 0xd455e0, 0xd455e0)
	/go/pkg/mod/go.etcd.io/[email protected]/bucket.go:181 +0x32d fp=0xc0000b29c8 sp=0xc0000b2810 pc=0x7f29ed
go.etcd.io/bbolt.(*Bucket).CreateBucketIfNotExists(0xc0001a60f8, 0xcea348, 0x2, 0x2, 0xc0001a60e0, 0xc0000b2a88, 0x43fe58)
	/go/pkg/mod/go.etcd.io/[email protected]/bucket.go:199 +0x5e fp=0xc0000b2a30 sp=0xc0000b29c8 pc=0x7f2e7e
go.etcd.io/bbolt.(*Tx).CreateBucketIfNotExists(...)
	/go/pkg/mod/go.etcd.io/[email protected]/tx.go:115
github.com/containerd/containerd/snapshots/storage.createBucketIfNotExists(0xab46c0, 0xc0001a40f0, 0xc0000b2b20, 0x1c9, 0x0)
	/go/pkg/mod/github.com/containerd/[email protected]/snapshots/storage/bolt.go:476 +0xc5 fp=0xc0000b2a98 sp=0xc0000b2a30 pc=0x821325
github.com/containerd/containerd/snapshots/storage.CreateSnapshot(0xab46c0, 0xc0001a40f0, 0xc0001a4002, 0xa28ad3, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/containerd/[email protected]/snapshots/storage/bolt.go:228 +0x256 fp=0xc0000b2b70 sp=0xc0000b2a98 pc=0x8202f6
github.com/containerd/stargz-snapshotter/snapshot.(*snapshotter).createSnapshot(0xc00009fec0, 0xab46c0, 0xc0001a40f0, 0xc00009b102, 0xa28ad3, 0x9, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/containerd/stargz-snapshotter/snapshot/snapshot.go:494 +0x69e fp=0xc0000b3158 sp=0xc0000b2b70 pc=0x95845e
github.com/containerd/stargz-snapshotter/snapshot.(*snapshotter).Prepare(0xc00009fec0, 0xab4680, 0xc0000a6018, 0xa28ad3, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/containerd/stargz-snapshotter/snapshot/snapshot.go:219 +0xd0 fp=0xc0000b3cb0 sp=0xc0000b3158 pc=0x954290
github.com/containerd/stargz-snapshotter/snapshot.TestOverlayMounts(0xc000083200)
	/go/src/github.com/containerd/stargz-snapshotter/snapshot/snapshot_test.go:466 +0x2d5 fp=0xc0000b3ed0 sp=0xc0000b3cb0 pc=0x960d95
testing.tRunner(0xc000083200, 0xa3bb30)
	/usr/local/go/src/testing/testing.go:1123 +0x203 fp=0xc0000b3fd0 sp=0xc0000b3ed0 pc=0x5caee3
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc0000b3fd8 sp=0xc0000b3fd0 pc=0x4a2fc1
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1168 +0x5bc

goroutine 1 [chan receive]:
testing.(*T).Run(0xc000082780, 0xa2bbd6, 0x11, 0xa3bb30, 0x1)
	/usr/local/go/src/testing/testing.go:1169 +0x5f4
testing.runTests.func1(0xc000082780)
	/usr/local/go/src/testing/testing.go:1439 +0xa7
testing.tRunner(0xc000082780, 0xc0000b1ce0)
	/usr/local/go/src/testing/testing.go:1123 +0x203
testing.runTests(0xc0000bf440, 0xd0fbe0, 0xa, 0xa, 0xbfee8bef3b937f52, 0x8bb364adcf, 0xd156c0, 0xc00017c4f0)
	/usr/local/go/src/testing/testing.go:1437 +0x613
testing.(*M).Run(0xc000134200, 0x0)
	/usr/local/go/src/testing/testing.go:1345 +0x3b4
main.main()
	_testmain.go:61 +0x237
FAIL	github.com/containerd/stargz-snapshotter/snapshot	0.023s

seems we need to bump up go.mod as well :cry;

ktock · 2020-12-16T07:29:03Z

In Tests / CRIValidation (pull_request) Failing after 4m — CRIValidation

The command '/bin/sh -c apt-get update && apt-get install -y iptables &&     update-alternatives --set iptables /usr/sbin/iptables-legacy &&     mkdir -p /opt/cni/bin &&     curl -Ls https://github.com/containernetworking/plugins/releases/download/${CNI_PLUGINS_VERSION}/cni-plugins-linux-${TARGETARCH}-${CNI_PLUGINS_VERSION}.tgz | tar xzv -C /opt/cni/bin' returned a non-zero code: 2

seems BuildKit (i.e. TARGETARCH ) isn't enabled on GitHub Actions ubuntu runner?

ktock · 2020-12-16T07:34:27Z

In Tests / PullSecrets (pull_request) and Tests / Optimize (pull_request)

ctr: Get "https://kind-private-registry:5000/v2/": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0

prepare_creds needs to be updated not to use Common Name field.

stargz-snapshotter/script/util/utils.sh

Lines 17 to 29 in af03a14

    
           # Preparing creds for provided user and password for private registry(only for testing purpose) 
        
           # See also: https://docs.docker.com/registry/deploying/ 
        
           function prepare_creds { 
        
               local OUTPUT="${1}" 
        
               local REGISTRY_HOST="${2}" 
        
               local USER="${3}" 
        
               local PASS="${4}" 
        
               mkdir "${OUTPUT}/auth" "${OUTPUT}/certs" 
        
               openssl req -subj "/C=JP/ST=Remote/L=Snapshotter/O=TestEnv/OU=Integration/CN=${REGISTRY_HOST}" \ 
        
                       -newkey rsa:2048 -nodes -keyout "${OUTPUT}/certs/domain.key" \ 
        
                       -x509 -days 365 -out "${OUTPUT}/certs/domain.crt" 
        
               htpasswd -Bbn "${USER}" "${PASS}" > "${OUTPUT}/auth/htpasswd" 
        
           }

Signed-off-by: Akihiro Suda <[email protected]>

Go 1.15 no longer accepts CN Signed-off-by: Akihiro Suda <[email protected]>

ktock · 2020-12-16T08:12:14Z

@AkihiroSuda In Benchmark / HelloBench (pull_request)

parse error: Invalid numeric literal at line 41, column 11

seems jq fails

stargz-snapshotter/script/util/utils.sh

Lines 41 to 42 in 55406f4

    
           REMOTE=$(jq -r 'select(."'"${LOG_REMOTE_SNAPSHOT}"'" == "true")' "${LOG_FILE}" | wc -l) 
        
           LOCAL=$(jq -r 'select(."'"${LOG_REMOTE_SNAPSHOT}"'" == "false")' "${LOG_FILE}" | wc -l)

because of garbage lines in the log:

2020/12/16 07:50:59 writer: Write/Writev failed, err: 2=no such file or directory. opcode: INTERRUPT

Please add options to jq there for ignoring non-json lines (c.f. jqlang/jq#884)

I'll fix it to make it more interigent in the following-up PRs.

AkihiroSuda · 2020-12-16T08:12:45Z

HeloBench failing

...
Pulling the image...
ctr-remote images rpull ghcr.io/stargz-containers/rethinkdb:2.3.6-esgz-noopt
ctr c create --net-host --snapshotter=stargz  -- ghcr.io/stargz-containers/rethinkdb:2.3.6-esgz-noopt rethinkdb-2.3.6_bench_918176 
ctr t start rethinkdb-2.3.6_bench_918176
out: Recursively removing directory /data/rethinkdb_data/tmp
out: Initializing directory /data/rethinkdb_data
out: Running rethinkdb 2.3.6~0jessie (GCC 4.9.2)...
out: Running on Linux 5.4.0-1031-azure x86_64
out: Loading data from directory /data/rethinkdb_data
out: Listening for intracluster connections on port 29015
out: Listening for client driver connections on port 28015
out: Listening for administrative HTTP connections on port 8080
out: Listening on cluster addresses: 127.0.0.1, 172.18.0.2
out: Listening on driver addresses: 127.0.0.1, 172.18.0.2
out: Listening on http addresses: 127.0.0.1, 172.18.0.2
out: Server ready, "79ca3578618a_j8c" ae1bd46e-7fc7-4a48-a9df-e876b3c51e90
DONE
Cleaning up environment...
ctr t kill -s 9 rethinkdb-2.3.6_bench_918176
ctr c rm rethinkdb-2.3.6_bench_918176
ctr image rm ghcr.io/stargz-containers/rethinkdb:2.3.6-esgz-noopt
/go/src/github.com/containerd/stargz-snapshotter/script/benchmark/hello-bench/src/../reboot_containerd.sh
ITERATION 0:
elapsed 5.47397494316
pull    1.44824004173
create  0.134563922882
run     2.75855398178
BENCHMARK_OUTPUT: {"elapsed_create": 0.13456392288208008, "elapsed_run": 2.758553981781006, "elapsed_pull": 1.448240041732788, "elapsed": 5.473974943161011, "repo": "rethinkdb:2.3.6", "mode": "estargz-noopt", "bench": "rethinkdb:2.3.6"},
parse error: Invalid numeric literal at line 41, column 11

ktock · 2020-12-16T08:15:43Z

Yes, that parse error: Invalid numeric literal at line 41, column 11 is because of #197 (comment).

AkihiroSuda · 2020-12-16T08:22:13Z

I guess this is an actually regression caused by bumping up Go. Not sure we should just ignore the error.

ktock · 2020-12-16T10:26:39Z

hanwen/go-fuse#375 seems related but needs more investigation.

Signed-off-by: Akihiro Suda <[email protected]>

- containerd: 1.4.2 -> 1.4.3 - CNI plugins: 0.8.6 -> 0.9.0 - kind node: 1.19.0 -> 1.20.0 Go is not updated in this commit. Signed-off-by: Akihiro Suda <[email protected]>

Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda · 2020-12-16T10:41:28Z

Removed Go 1.15 from this PR. Let's see whether it can pass the CI..

AkihiroSuda · 2020-12-16T14:52:46Z

CI passing. Can we bump Go in a separate PR?

ktock · 2020-12-16T14:59:13Z

CI passing. Can we bump Go in a separate PR?

Yes, let's do so.

AkihiroSuda requested a review from ktock December 16, 2020 05:54

AkihiroSuda force-pushed the bump-dockerfile-20201216 branch from 921d77a to 45ccc8c Compare December 16, 2020 05:55

AkihiroSuda changed the title ~~Dockerfile: bump up components~~ Dockerfile: bump up components (Go 1.15, Kubernetes 1.20, ...) Dec 16, 2020

AkihiroSuda force-pushed the bump-dockerfile-20201216 branch 2 times, most recently from b02cf69 to 7f27b4a Compare December 16, 2020 07:21

AkihiroSuda added 3 commits December 16, 2020 16:45

go.mod: bump up containerd to support Go 1.15

5bb0409

Signed-off-by: Akihiro Suda <[email protected]>

GHA: enable BuildKit globally

126f6c3

Signed-off-by: Akihiro Suda <[email protected]>

script/util: use SAN for generating TLS certs

2bbc32e

Go 1.15 no longer accepts CN Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda force-pushed the bump-dockerfile-20201216 branch from 1c1a278 to a2dee46 Compare December 16, 2020 07:46

AkihiroSuda added 3 commits December 16, 2020 19:37

go.mod: use go-fuse@master

ae189d4

Signed-off-by: Akihiro Suda <[email protected]>

Dockerfile: bump up components

a7cd58a

- containerd: 1.4.2 -> 1.4.3 - CNI plugins: 0.8.6 -> 0.9.0 - kind node: 1.19.0 -> 1.20.0 Go is not updated in this commit. Signed-off-by: Akihiro Suda <[email protected]>

script: go install -> go get

ab957d6

Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda force-pushed the bump-dockerfile-20201216 branch from 7c9db01 to ab957d6 Compare December 16, 2020 10:40

AkihiroSuda changed the title ~~Dockerfile: bump up components (Go 1.15, Kubernetes 1.20, ...)~~ Dockerfile: bump up components (Kubernetes 1.20, ...) Dec 16, 2020

ktock approved these changes Dec 16, 2020

View reviewed changes

ktock merged commit a7f43f4 into containerd:master Dec 16, 2020

AkihiroSuda mentioned this pull request Dec 16, 2020

update Go to 1.15 #202

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Dockerfile: bump up components (Kubernetes 1.20, ...) #197

Dockerfile: bump up components (Kubernetes 1.20, ...) #197

Uh oh!

AkihiroSuda commented Dec 16, 2020 •

edited

Loading

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Dockerfile: bump up components (Kubernetes 1.20, ...) #197

Dockerfile: bump up components (Kubernetes 1.20, ...) #197

Uh oh!

Conversation

AkihiroSuda commented Dec 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

AkihiroSuda commented Dec 16, 2020

Uh oh!

ktock commented Dec 16, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AkihiroSuda commented Dec 16, 2020 •

edited

Loading