PR #109284 pull-kubernetes-node-e2e-containerd #1526452412997439488
Recent runs || View in Spyglass
| PR | wojtek-t: [WIP] [DO NOT REVIEW] Debug leaking goroutines |
| Result | FAILURE |
| Tests | 1 failed / 9 succeeded |
| Started | |
| Elapsed | 8m18s |
| Revision | |
| Builder | 26976b1e-d5ac-11ec-8f17-46344a586d8e |
| Refs |
master:c84d0864 109284:dd27dc34 |
| infra-commit | da9cf8a51 |
| job-version | v1.25.0-alpha.0.552+0b168c9e43c4d9 |
| kubetest-version | |
| repo | k8s.io/kubernetes |
| repo-commit | 0b168c9e43c4d936c3bb84ca539d8c435729bd9d |
| repos | {u'k8s.io/kubernetes': u'master:c84d0864ddebbd0d36ce295cf74a447f24d7b3ec,109284:dd27dc348f30acb518ab58c4611a9e010bd09c23'} |
| revision | v1.25.0-alpha.0.552+0b168c9e43c4d9 |
error during go run /go/src/k8s.io/kubernetes/test/e2e_node/runner/remote/run_remote.go --cleanup --logtostderr --vmodule=*=4 --ssh-env=gce --results-dir=/workspace/_artifacts --project=k8s-infra-e2e-boskos-075 --zone=us-west1-b --ssh-user=prow --ssh-key=/workspace/.ssh/google_compute_engine --ginkgo-flags=--nodes=8 --focus="\[NodeConformance\]" --skip="\[Flaky\]|\[Slow\]|\[Serial\]" --test_args=--container-runtime-endpoint=unix:///run/containerd/containerd.sock --container-runtime-process-name=/usr/bin/containerd --container-runtime-pid-file= --kubelet-flags="--cgroups-per-qos=true --cgroup-root=/ --runtime-cgroups=/system.slice/containerd.service" --extra-log="{\"name\": \"containerd.log\", \"journalctl\": [\"-u\", \"containerd*\"]}" --test-timeout=1h5m0s --image-config-file=/workspace/test-infra/jobs/e2e_node/containerd/image-config.yaml: exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest Deferred TearDown
kubetest DumpClusterLogs
kubetest GetDeployer
kubetest Prepare
kubetest TearDown
kubetest TearDown Previous
kubetest Timeout
kubetest Up
kubetest test setup
... skipping 326 lines ... W0517 06:45:54.242] - ctr version # current version of containerd W0517 06:45:54.242] W0517 06:45:54.242] - echo "Download and install CNI configuration to /home/containerd" W0517 06:45:54.242] - mkdir -p /home/containerd W0517 06:45:54.242] - mount --bind /home/containerd /home/containerd W0517 06:45:54.242] - mount -o remount,exec /home/containerd W0517 06:45:54.242] - 'curl --fail --retry 5 --retry-delay 3 --silent --show-error -H "X-Google-Metadata-Request: True" -o /home/containerd/cni.template http://metadata.google.internal/computeMetadata/v1/instance/attributes/cni-template' W0517 06:45:54.243] W0517 06:45:54.243] - echo "Download and install CNI to /home/containerd" W0517 06:45:54.243] - 'curl -sSL --fail --retry 5 --retry-delay 3 --silent --show-error -o /home/containerd/cni.tgz https://github.com/containernetworking/plugins/releases/download/v1.0.1/cni-plugins-linux-amd64-v1.0.1.tgz' W0517 06:45:54.243] - tar xzf /home/containerd/cni.tgz -C /home/containerd --overwrite W0517 06:45:54.243] W0517 06:45:54.243] - echo "Set containerd configuration" W0517 06:45:54.243] - mkdir -p /etc/containerd W0517 06:45:54.244] - 'curl --fail --retry 5 --retry-delay 3 --silent --show-error -H "X-Google-Metadata-Request: True" -o /etc/containerd/config.toml http://metadata.google.internal/computeMetadata/v1/instance/attributes/containerd-config' W0517 06:45:54.244] W0517 06:45:54.244] - echo "Restarting containerd" W0517 06:45:54.244] - systemctl restart containerd W0517 06:45:54.244] W0517 06:45:54.244] - echo "Configuration complete" W0517 06:45:54.245] ] ... skipping 66 lines ... W0517 06:45:55.702] - ctr version # current version of containerd W0517 06:45:55.703] W0517 06:45:55.703] - echo "Download and install CNI configuration to /home/containerd" W0517 06:45:55.703] - mkdir -p /home/containerd W0517 06:45:55.703] - mount --bind /home/containerd /home/containerd W0517 06:45:55.703] - mount -o remount,exec /home/containerd W0517 06:45:55.704] - 'curl --fail --retry 5 --retry-delay 3 --silent --show-error -H "X-Google-Metadata-Request: True" -o /home/containerd/cni.template http://metadata.google.internal/computeMetadata/v1/instance/attributes/cni-template' W0517 06:45:55.704] W0517 06:45:55.704] - echo "Download and install CNI to /home/containerd" W0517 06:45:55.704] - 'curl -sSL --fail --retry 5 --retry-delay 3 --silent --show-error -o /home/containerd/cni.tgz https://github.com/containernetworking/plugins/releases/download/v1.0.1/cni-plugins-linux-amd64-v1.0.1.tgz' W0517 06:45:55.704] - tar xzf /home/containerd/cni.tgz -C /home/containerd --overwrite W0517 06:45:55.705] W0517 06:45:55.705] - echo "Set containerd configuration" W0517 06:45:55.705] - mkdir -p /etc/containerd W0517 06:45:55.706] - 'curl --fail --retry 5 --retry-delay 3 --silent --show-error -H "X-Google-Metadata-Request: True" -o /etc/containerd/config.toml http://metadata.google.internal/computeMetadata/v1/instance/attributes/containerd-config' W0517 06:45:55.706] W0517 06:45:55.706] - echo "Restarting containerd" W0517 06:45:55.706] - systemctl restart containerd W0517 06:45:55.706] W0517 06:45:55.707] - echo "Configuration complete" W0517 06:45:55.707] ] ... skipping 7 lines ... W0517 06:45:55.909] I0517 06:45:55.836909 7030 run_remote.go:596] Creating instance {image:ubuntu-gke-2004-1-23-v20220511 imageDesc:ubuntu-gke-2004-1-23-v20220511 kernelArguments:[] project:ubuntu-os-gke-cloud resources:{Accelerators:[]} metadata:0xc00073ed90 machine: tests:[]} with service account "976069798193-compute@developer.gserviceaccount.com" I0517 06:45:57.076] +++ [0517 06:45:57] Building go targets for linux/amd64 I0517 06:45:57.095] k8s.io/kubernetes/hack/make-rules/helpers/go2make (non-static) I0517 06:46:10.989] +++ [0517 06:46:10] Building go targets for linux/amd64 I0517 06:46:11.006] k8s.io/code-generator/cmd/prerelease-lifecycle-gen (non-static) I0517 06:46:16.918] +++ [0517 06:46:16] Generating prerelease lifecycle code for 26 targets W0517 06:46:17.475] I0517 06:46:17.474555 7030 ssh.go:120] Running the command ssh, with args: [-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o CheckHostIP=no -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o LogLevel=ERROR -i /workspace/.ssh/google_compute_engine prow@34.145.107.34 -- sudo sh -c 'systemctl list-units --type=service --state=running | grep -e containerd -e crio'] W0517 06:46:17.520] E0517 06:46:17.520109 7030 ssh.go:123] failed to run SSH command: out: ssh: connect to host 34.145.107.34 port 22: Connection refused W0517 06:46:17.520] , err: exit status 255 W0517 06:46:17.733] I0517 06:46:17.732756 7030 ssh.go:120] Running the command ssh, with args: [-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o CheckHostIP=no -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o LogLevel=ERROR -i /workspace/.ssh/google_compute_engine prow@34.105.104.238 -- sudo sh -c 'systemctl list-units --type=service --state=running | grep -e containerd -e crio'] I0517 06:46:19.393] +++ [0517 06:46:19] Building go targets for linux/amd64 I0517 06:46:19.410] k8s.io/code-generator/cmd/deepcopy-gen (non-static) I0517 06:46:21.445] +++ [0517 06:46:21] Generating deepcopy code for 236 targets W0517 06:46:21.825] I0517 06:46:21.824445 7030 ssh.go:120] Running the command ssh, with args: [-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o CheckHostIP=no -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o LogLevel=ERROR -i /workspace/.ssh/google_compute_engine prow@34.105.104.238 -- sudo ls /var/lib/cloud/instance/boot-finished] W0517 06:46:22.421] E0517 06:46:22.421429 7030 ssh.go:123] failed to run SSH command: out: ls: cannot access '/var/lib/cloud/instance/boot-finished': No such file or directory W0517 06:46:22.422] , err: exit status 2 I0517 06:46:28.917] +++ [0517 06:46:28] Building go targets for linux/amd64 I0517 06:46:28.935] k8s.io/code-generator/cmd/defaulter-gen (non-static) I0517 06:46:30.220] +++ [0517 06:46:30] Generating defaulter code for 92 targets W0517 06:46:37.783] I0517 06:46:37.782701 7030 ssh.go:120] Running the command ssh, with args: [-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o CheckHostIP=no -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o LogLevel=ERROR -i /workspace/.ssh/google_compute_engine prow@34.145.107.34 -- sudo sh -c 'systemctl list-units --type=service --state=running | grep -e containerd -e crio'] W0517 06:46:39.043] I0517 06:46:39.043437 7030 ssh.go:120] Running the command ssh, with args: [-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o CheckHostIP=no -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o LogLevel=ERROR -i /workspace/.ssh/google_compute_engine prow@34.145.107.34 -- sudo ls /var/lib/cloud/instance/boot-finished] I0517 06:46:40.845] +++ [0517 06:46:40] Building go targets for linux/amd64 I0517 06:46:40.863] k8s.io/code-generator/cmd/conversion-gen (non-static) W0517 06:46:42.423] I0517 06:46:42.423013 7030 ssh.go:120] Running the command ssh, with args: [-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o CheckHostIP=no -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o LogLevel=ERROR -i /workspace/.ssh/google_compute_engine prow@34.105.104.238 -- sudo ls /var/lib/cloud/instance/boot-finished] I0517 06:46:42.523] +++ [0517 06:46:42] Generating conversion code for 129 targets I0517 06:47:04.136] +++ [0517 06:47:04] Building go targets for linux/amd64 I0517 06:47:04.154] k8s.io/kube-openapi/cmd/openapi-gen (non-static) I0517 06:47:13.562] +++ [0517 06:47:13] Generating openapi code for KUBE I0517 06:47:35.694] +++ [0517 06:47:35] Generating openapi code for AGGREGATOR I0517 06:47:37.366] +++ [0517 06:47:37] Generating openapi code for APIEXTENSIONS ... skipping 13 lines ... W0517 06:50:59.526] !!! [0517 06:50:59] 2: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:888 kube::golang::build_binaries_for_platform(...) W0517 06:50:59.528] !!! [0517 06:50:59] 3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) W0517 06:50:59.532] !!! [0517 06:50:59] Call tree: W0517 06:50:59.534] !!! [0517 06:50:59] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) W0517 06:50:59.538] !!! [0517 06:50:59] Call tree: W0517 06:50:59.540] !!! [0517 06:50:59] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) W0517 06:50:59.540] make: *** [Makefile:94: all] Error 1 I0517 06:50:59.641] make: Leaving directory '/go/src/k8s.io/kubernetes' W0517 06:50:59.812] I0517 06:50:59.812723 7030 run_remote.go:872] Deleting instance "tmp-node-e2e-4f06ef7e-cos-97-16919-29-21" W0517 06:50:59.827] I0517 06:50:59.827652 7030 run_remote.go:872] Deleting instance "tmp-node-e2e-4f06ef7e-ubuntu-gke-2004-1-23-v20220511" I0517 06:51:00.277] I0517 06:51:00.277] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I0517 06:51:00.277] > START TEST > I0517 06:51:00.278] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I0517 06:51:00.278] Start Test Suite on Host I0517 06:51:00.278] I0517 06:51:00.278] Failure Finished Test Suite on Host I0517 06:51:00.278] unable to create test archive: failed to setup test package "/tmp/node-e2e-archive4027948537": failed to build the dependencies: failed to build go packages exit status 2 I0517 06:51:00.278] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I0517 06:51:00.279] < FINISH TEST < I0517 06:51:00.279] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I0517 06:51:00.279] I0517 06:51:00.285] I0517 06:51:00.285] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I0517 06:51:00.285] > START TEST > I0517 06:51:00.285] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I0517 06:51:00.286] Start Test Suite on Host I0517 06:51:00.286] I0517 06:51:00.286] Failure Finished Test Suite on Host I0517 06:51:00.286] unable to create test archive: failed to setup test package "/tmp/node-e2e-archive4027948537": failed to build the dependencies: failed to build go packages exit status 2 I0517 06:51:00.286] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I0517 06:51:00.287] < FINISH TEST < I0517 06:51:00.287] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I0517 06:51:00.287] I0517 06:51:00.287] Failure: 2 errors encountered. W0517 06:51:00.388] exit status 1 ... skipping 11 lines ... I0517 06:51:00.669] Sourcing kube-util.sh I0517 06:51:00.669] Detecting project I0517 06:51:00.669] Project: k8s-infra-e2e-boskos-075 I0517 06:51:00.669] Network Project: k8s-infra-e2e-boskos-075 I0517 06:51:00.669] Zone: us-west1-b I0517 06:51:00.670] Dumping logs from master locally to '/workspace/_artifacts' W0517 06:51:01.516] ERROR: (gcloud.compute.addresses.describe) Could not fetch resource: W0517 06:51:01.517] - The resource 'projects/k8s-infra-e2e-boskos-075/regions/us-west1/addresses/bootstrap-e2e-master-ip' was not found W0517 06:51:01.517] W0517 06:51:01.646] Could not detect Kubernetes master node. Make sure you've launched a cluster with 'kube-up.sh' I0517 06:51:01.746] Master not detected. Is the cluster up? I0517 06:51:01.747] Dumping logs from nodes locally to '/workspace/_artifacts' I0517 06:51:01.747] Detecting nodes in the cluster ... skipping 4 lines ... W0517 06:51:06.760] NODE_NAMES= W0517 06:51:06.762] 2022/05/17 06:51:06 process.go:155: Step './cluster/log-dump/log-dump.sh /workspace/_artifacts' finished in 6.265126592s W0517 06:51:06.762] 2022/05/17 06:51:06 node.go:53: Noop - Node Down() W0517 06:51:06.762] 2022/05/17 06:51:06 process.go:96: Saved XML output to /workspace/_artifacts/junit_runner.xml. W0517 06:51:06.762] 2022/05/17 06:51:06 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}" W0517 06:51:07.147] 2022/05/17 06:51:07 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 385.441646ms W0517 06:51:07.162] 2022/05/17 06:51:07 main.go:331: Something went wrong: encountered 1 errors: [error during go run /go/src/k8s.io/kubernetes/test/e2e_node/runner/remote/run_remote.go --cleanup --logtostderr --vmodule=*=4 --ssh-env=gce --results-dir=/workspace/_artifacts --project=k8s-infra-e2e-boskos-075 --zone=us-west1-b --ssh-user=prow --ssh-key=/workspace/.ssh/google_compute_engine --ginkgo-flags=--nodes=8 --focus="\[NodeConformance\]" --skip="\[Flaky\]|\[Slow\]|\[Serial\]" --test_args=--container-runtime-endpoint=unix:///run/containerd/containerd.sock --container-runtime-process-name=/usr/bin/containerd --container-runtime-pid-file= --kubelet-flags="--cgroups-per-qos=true --cgroup-root=/ --runtime-cgroups=/system.slice/containerd.service" --extra-log="{\"name\": \"containerd.log\", \"journalctl\": [\"-u\", \"containerd*\"]}" --test-timeout=1h5m0s --image-config-file=/workspace/test-infra/jobs/e2e_node/containerd/image-config.yaml: exit status 1] W0517 06:51:07.163] Traceback (most recent call last): W0517 06:51:07.163] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 723, in <module> W0517 06:51:07.163] main(parse_args()) W0517 06:51:07.164] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 569, in main W0517 06:51:07.164] mode.start(runner_args) W0517 06:51:07.164] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 228, in start W0517 06:51:07.164] check_env(env, self.command, *args) W0517 06:51:07.164] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 111, in check_env W0517 06:51:07.164] subprocess.check_call(cmd, env=env) W0517 06:51:07.165] File "/usr/lib/python2.7/subprocess.py", line 190, in check_call W0517 06:51:07.165] raise CalledProcessError(retcode, cmd) W0517 06:51:07.166] subprocess.CalledProcessError: Command '('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--test', '--deployment=node', '--provider=gce', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--gcp-zone=us-west1-b', '--node-test-args=--container-runtime-endpoint=unix:///run/containerd/containerd.sock --container-runtime-process-name=/usr/bin/containerd --container-runtime-pid-file= --kubelet-flags="--cgroups-per-qos=true --cgroup-root=/ --runtime-cgroups=/system.slice/containerd.service" --extra-log="{\\"name\\": \\"containerd.log\\", \\"journalctl\\": [\\"-u\\", \\"containerd*\\"]}"', '--node-tests=true', '--test_args=--nodes=8 --focus="\\[NodeConformance\\]" --skip="\\[Flaky\\]|\\[Slow\\]|\\[Serial\\]"', '--timeout=65m', '--node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/containerd/image-config.yaml')' returned non-zero exit status 1 E0517 06:51:07.170] Command failed I0517 06:51:07.170] process 535 exited with code 1 after 6.8m E0517 06:51:07.170] FAIL: pull-kubernetes-node-e2e-containerd I0517 06:51:07.171] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W0517 06:51:07.889] Activated service account credentials for: [prow-build@k8s-infra-prow-build.iam.gserviceaccount.com] I0517 06:51:07.982] process 26278 exited with code 0 after 0.0m I0517 06:51:07.982] Call: gcloud config get-value account I0517 06:51:08.612] process 26292 exited with code 0 after 0.0m I0517 06:51:08.612] Will upload results to gs://kubernetes-jenkins/pr-logs using prow-build@k8s-infra-prow-build.iam.gserviceaccount.com I0517 06:51:08.612] Upload result and artifacts... I0517 06:51:08.613] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/pr-logs/pull/109284/pull-kubernetes-node-e2e-containerd/1526452412997439488 I0517 06:51:08.613] Call: gsutil ls gs://kubernetes-jenkins/pr-logs/pull/109284/pull-kubernetes-node-e2e-containerd/1526452412997439488/artifacts W0517 06:51:09.770] CommandException: One or more URLs matched no objects. E0517 06:51:09.944] Command failed I0517 06:51:09.944] process 26306 exited with code 1 after 0.0m W0517 06:51:09.945] Remote dir gs://kubernetes-jenkins/pr-logs/pull/109284/pull-kubernetes-node-e2e-containerd/1526452412997439488/artifacts not exist yet I0517 06:51:09.945] Call: gsutil -m -q -o GSUtil:use_magicfile=True cp -r -c -z log,txt,xml /workspace/_artifacts gs://kubernetes-jenkins/pr-logs/pull/109284/pull-kubernetes-node-e2e-containerd/1526452412997439488/artifacts I0517 06:51:11.692] process 26446 exited with code 0 after 0.0m I0517 06:51:11.693] Call: git rev-parse HEAD I0517 06:51:11.696] process 26962 exited with code 0 after 0.0m ... skipping 21 lines ...