This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 1 failed / 9 succeeded
Started2020-03-20 13:00
Elapsed21m16s
Revisionrelease-1.16
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/5b86a756-1b83-4f49-a92a-01bf1d8ebeba/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/5b86a756-1b83-4f49-a92a-01bf1d8ebeba/targets/test
uploadercrier

Test Failures


task-06-upgrade 5m0s

timeout. task did not completed in less than 5m0s as expected
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 9 Passed Tests

Show 3 Skipped Tests

Error lines from build-log.txt

... skipping 285 lines ...
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /kind/systemd/kubelet.service.
Created symlink /etc/systemd/system/kubelet.service → /kind/systemd/kubelet.service.
time="13:06:36" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e mkdir -p /etc/systemd/system/kubelet.service.d]"
time="13:06:37" level=info msg="Adding /etc/systemd/system/kubelet.service.d/10-kubeadm.conf to the image"
time="13:06:37" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e cp /alter/bits/systemd/10-kubeadm.conf /etc/systemd/system/kubelet.service.d/10-kubeadm.conf]"
time="13:06:37" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e chown -R root:root /etc/systemd/system/kubelet.service.d/10-kubeadm.conf]"
time="13:06:38" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e /bin/sh -c echo \"KUBELET_EXTRA_ARGS=--fail-swap-on=false\" >> /etc/default/kubelet]"
time="13:06:38" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e mkdir -p /kinder]"
time="13:06:39" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e rsync -r /alter/bits/upgrade /kinder]"
time="13:06:53" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e chown -R root:root /kinder/upgrade]"
time="13:06:54" level=debug msg="Running: [docker exec kind-build-3b10c0da-9fff-4839-a8a1-b28c3bd1c17e /bin/sh -c which docker || true]"
time="13:06:55" level=info msg="Detected docker as container runtime"
time="13:06:55" level=info msg="Pre loading images ..."
... skipping 172 lines ...
kinder-upgrade-control-plane-1:$ Preparing /kind/kubeadm.conf
time="13:08:59" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-1]"
time="13:08:59" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-1 kubeadm version -o=short]"
time="13:09:00" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:09:00" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:09:00" level=debug msg="Preparing automaticCopyCertsPatches for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:09:00" level=debug msg="generated config:\napiServer:\n  certSANs:\n  - localhost\n  - 172.17.0.3\napiVersion: kubeadm.k8s.io/v1beta2\nclusterName: kinder-upgrade\ncontrolPlaneEndpoint: 172.17.0.7:6443\ncontrollerManager:\n  extraArgs:\n    enable-hostpath-provisioner: \"true\"\nkind: ClusterConfiguration\nkubernetesVersion: v1.15.12-beta.0.7+7f18f85e0e8bcc\nnetworking:\n  podSubnet: 192.168.0.0/16\n  serviceSubnet: \"\"\nscheduler:\n  extraArgs: null\n---\napiVersion: kubeadm.k8s.io/v1beta2\nbootstrapTokens:\n- token: abcdef.0123456789abcdef\ncertificateKey: \"0123456789012345678901234567890123456789012345678901234567890123\"\nkind: InitConfiguration\nlocalAPIEndpoint:\n  advertiseAddress: 172.17.0.3\n  bindPort: 6443\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.3\n---\napiVersion: kubelet.config.k8s.io/v1beta1\nevictionHard:\n  imagefs.available: 0%\n  nodefs.available: 0%\n  nodefs.inodesFree: 0%\nimageGCHighThresholdPercent: 100\nkind: KubeletConfiguration\n---\napiVersion: kubeproxy.config.k8s.io/v1alpha1\nkind: KubeProxyConfiguration\n"
time="13:09:00" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-control-plane-1-631714070 kinder-upgrade-control-plane-1:/kind/kubeadm.conf]"

kinder-upgrade-lb:$ Updating load balancer configuration with 1 control plane backends
time="13:09:01" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-1]"
time="13:09:01" level=debug msg="Writing loadbalancer config on kinder-upgrade-lb..."
time="13:09:01" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-lb-601464189 kinder-upgrade-lb:/usr/local/etc/haproxy/haproxy.cfg]"
... skipping 32 lines ...
I0320 13:09:05.493765     563 checks.go:382] validating the presence of executable ebtables
I0320 13:09:05.493839     563 checks.go:382] validating the presence of executable ethtool
I0320 13:09:05.493900     563 checks.go:382] validating the presence of executable socat
I0320 13:09:05.493956     563 checks.go:382] validating the presence of executable tc
I0320 13:09:05.494032     563 checks.go:382] validating the presence of executable touch
I0320 13:09:05.494094     563 checks.go:524] running all checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0320 13:09:05.589645     563 checks.go:412] checking whether the given node name is reachable using net.LookupHost
I0320 13:09:05.589933     563 checks.go:622] validating kubelet version
I0320 13:09:05.743956     563 checks.go:131] validating if the service is enabled and active
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
... skipping 399 lines ...
kinder-upgrade-control-plane-2:$ Preparing /kind/kubeadm.conf
time="13:11:23" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-2]"
time="13:11:24" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-2 kubeadm version -o=short]"
time="13:11:24" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:11:24" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:11:24" level=debug msg="Preparing automaticCopyCertsPatches for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:11:24" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ncontrolPlane:\n  certificateKey: \"0123456789012345678901234567890123456789012345678901234567890123\"\n  localAPIEndpoint:\n    advertiseAddress: 172.17.0.4\n    bindPort: 6443\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.4\n"
time="13:11:24" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-control-plane-2-344818139 kinder-upgrade-control-plane-2:/kind/kubeadm.conf]"
time="13:11:25" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-2 kubeadm version -o=short]"

kinder-upgrade-control-plane-2:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="13:11:26" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-2 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
I0320 13:11:27.350710     699 join.go:364] [preflight] found NodeName empty; using OS hostname as NodeName
... skipping 17 lines ...
I0320 13:11:27.947519     699 checks.go:382] validating the presence of executable ebtables
I0320 13:11:27.947552     699 checks.go:382] validating the presence of executable ethtool
I0320 13:11:27.947584     699 checks.go:382] validating the presence of executable socat
I0320 13:11:27.947613     699 checks.go:382] validating the presence of executable tc
I0320 13:11:27.947646     699 checks.go:382] validating the presence of executable touch
I0320 13:11:27.947714     699 checks.go:524] running all checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0320 13:11:28.034642     699 checks.go:412] checking whether the given node name is reachable using net.LookupHost
I0320 13:11:28.034946     699 checks.go:622] validating kubelet version
I0320 13:11:28.164989     699 checks.go:131] validating if the service is enabled and active
I0320 13:11:28.198484     699 checks.go:209] validating availability of port 10250
I0320 13:11:28.198722     699 checks.go:439] validating if the connectivity type is via proxy or direct
I0320 13:11:28.198767     699 join.go:433] [preflight] Discovering cluster-info
... skipping 192 lines ...
kinder-upgrade-control-plane-3:$ Preparing /kind/kubeadm.conf
time="13:12:27" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-3]"
time="13:12:27" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-3 kubeadm version -o=short]"
time="13:12:28" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:12:28" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:12:28" level=debug msg="Preparing automaticCopyCertsPatches for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:12:28" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ncontrolPlane:\n  certificateKey: \"0123456789012345678901234567890123456789012345678901234567890123\"\n  localAPIEndpoint:\n    advertiseAddress: 172.17.0.6\n    bindPort: 6443\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.6\n"
time="13:12:28" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-control-plane-3-360528069 kinder-upgrade-control-plane-3:/kind/kubeadm.conf]"
time="13:12:29" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-3 kubeadm version -o=short]"

kinder-upgrade-control-plane-3:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="13:12:30" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-3 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
I0320 13:12:30.803931     785 join.go:364] [preflight] found NodeName empty; using OS hostname as NodeName
... skipping 17 lines ...
I0320 13:12:31.447220     785 checks.go:382] validating the presence of executable ebtables
I0320 13:12:31.447577     785 checks.go:382] validating the presence of executable ethtool
I0320 13:12:31.447625     785 checks.go:382] validating the presence of executable socat
I0320 13:12:31.447709     785 checks.go:382] validating the presence of executable tc
I0320 13:12:31.447802     785 checks.go:382] validating the presence of executable touch
I0320 13:12:31.447864     785 checks.go:524] running all checks
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0320 13:12:31.524187     785 checks.go:412] checking whether the given node name is reachable using net.LookupHost
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
... skipping 201 lines ...

kinder-upgrade-worker-1:$ Preparing /kind/kubeadm.conf
time="13:13:30" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-worker-1]"
time="13:13:30" level=debug msg="Running: [docker exec kinder-upgrade-worker-1 kubeadm version -o=short]"
time="13:13:31" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:13:31" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:13:31" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.2\n"
time="13:13:31" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-worker-1-315587903 kinder-upgrade-worker-1:/kind/kubeadm.conf]"

kinder-upgrade-worker-1:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="13:13:32" level=debug msg="Running: [docker exec kinder-upgrade-worker-1 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
I0320 13:13:33.011154     826 join.go:364] [preflight] found NodeName empty; using OS hostname as NodeName
I0320 13:13:33.011199     826 joinconfiguration.go:75] loading configuration from "/kind/kubeadm.conf"
... skipping 16 lines ...
I0320 13:13:33.617472     826 checks.go:382] validating the presence of executable ebtables
I0320 13:13:33.617520     826 checks.go:382] validating the presence of executable ethtool
I0320 13:13:33.617582     826 checks.go:382] validating the presence of executable socat
I0320 13:13:33.617624     826 checks.go:382] validating the presence of executable tc
I0320 13:13:33.617659     826 checks.go:382] validating the presence of executable touch
I0320 13:13:33.617700     826 checks.go:524] running all checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0320 13:13:33.702797     826 checks.go:412] checking whether the given node name is reachable using net.LookupHost
I0320 13:13:33.703144     826 checks.go:622] validating kubelet version
I0320 13:13:33.893104     826 checks.go:131] validating if the service is enabled and active
I0320 13:13:33.937531     826 checks.go:209] validating availability of port 10250
I0320 13:13:33.937801     826 checks.go:292] validating the existence of file /etc/kubernetes/pki/ca.crt
I0320 13:13:33.937825     826 checks.go:439] validating if the connectivity type is via proxy or direct
... skipping 85 lines ...

kinder-upgrade-worker-2:$ Preparing /kind/kubeadm.conf
time="13:14:13" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-worker-2]"
time="13:14:13" level=debug msg="Running: [docker exec kinder-upgrade-worker-2 kubeadm version -o=short]"
time="13:14:14" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:14:14" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.15.12-beta.0.7+7f18f85e0e8bcc)"
time="13:14:14" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.5\n"
time="13:14:14" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-worker-2-403073426 kinder-upgrade-worker-2:/kind/kubeadm.conf]"

kinder-upgrade-worker-2:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="13:14:15" level=debug msg="Running: [docker exec kinder-upgrade-worker-2 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
I0320 13:14:15.722427     859 join.go:364] [preflight] found NodeName empty; using OS hostname as NodeName
I0320 13:14:15.722469     859 joinconfiguration.go:75] loading configuration from "/kind/kubeadm.conf"
... skipping 16 lines ...
I0320 13:14:16.505118     859 checks.go:382] validating the presence of executable ebtables
I0320 13:14:16.505520     859 checks.go:382] validating the presence of executable ethtool
I0320 13:14:16.505625     859 checks.go:382] validating the presence of executable socat
I0320 13:14:16.505681     859 checks.go:382] validating the presence of executable tc
I0320 13:14:16.505724     859 checks.go:382] validating the presence of executable touch
I0320 13:14:16.505803     859 checks.go:524] running all checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0320 13:14:16.613463     859 checks.go:412] checking whether the given node name is reachable using net.LookupHost
I0320 13:14:16.614089     859 checks.go:622] validating kubelet version
I0320 13:14:16.811058     859 checks.go:131] validating if the service is enabled and active
I0320 13:14:16.846591     859 checks.go:209] validating availability of port 10250
I0320 13:14:16.846853     859 checks.go:292] validating the existence of file /etc/kubernetes/pki/ca.crt
I0320 13:14:16.846881     859 checks.go:439] validating if the connectivity type is via proxy or direct
... skipping 1191 lines ...
I0320 13:20:06.519650   19425 local.go:106] [etcd] get the member id from peer: https://172.17.0.3:2380
I0320 13:20:06.545582   19425 local.go:112] [etcd] removing etcd member: https://172.17.0.3:2380, id: 12858828581462913056
I0320 13:20:06.573969   19425 local.go:117] [etcd] Updated etcd member list: [{kinder-upgrade-control-plane-2 https://172.17.0.4:2380} {kinder-upgrade-control-plane-3 https://172.17.0.6:2380}]
I0320 13:20:06.574014   19425 cleanupnode.go:57] [reset] Getting init system
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0320 13:20:06.727097   19425 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0320 13:20:06.834964   19425 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
I0320 13:20:22.730003   19425 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/etcd /var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
I0320 13:20:22.731686   19425 reset.go:209] [reset] Deleting content of /var/lib/etcd
... skipping 38 lines ...
I0320 13:20:23.873384   11856 local.go:106] [etcd] get the member id from peer: https://172.17.0.4:2380
I0320 13:20:23.896254   11856 local.go:112] [etcd] removing etcd member: https://172.17.0.4:2380, id: 14587267851092868664
I0320 13:20:23.923031   11856 local.go:117] [etcd] Updated etcd member list: [{kinder-upgrade-control-plane-3 https://172.17.0.6:2380}]
I0320 13:20:23.923084   11856 cleanupnode.go:57] [reset] Getting init system
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0320 13:20:24.088151   11856 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0320 13:20:24.197047   11856 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
I0320 13:20:39.046621   11856 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/etcd /var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
I0320 13:20:39.048113   11856 reset.go:209] [reset] Deleting content of /var/lib/etcd
... skipping 30 lines ...
I0320 13:20:40.117655   10377 uploadconfig.go:58] Removing APIEndpoint for Node "kinder-upgrade-control-plane-3"
I0320 13:20:40.122510   10377 round_trippers.go:443] PUT https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config 200 OK in 4 milliseconds
I0320 13:20:40.122647   10377 removeetcdmember.go:54] [reset] Checking for etcd config
I0320 13:20:40.122670   10377 local.go:97] [etcd] creating etcd client that connects to etcd pods
I0320 13:20:40.130526   10377 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config 200 OK in 7 milliseconds
I0320 13:20:40.130948   10377 etcd.go:107] etcd endpoints read from pods: 
W0320 13:20:40.131589   10377 removeetcdmember.go:61] [reset] failed to remove etcd member: error syncing endpoints with etc: etcdclient: no available endpoints
.Please manually remove this etcd member using etcdctl
I0320 13:20:40.131643   10377 cleanupnode.go:57] [reset] Getting init system
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0320 13:20:40.250050   10377 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0320 13:20:40.316385   10377 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
I0320 13:20:52.094938   10377 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
I0320 13:20:52.096914   10377 reset.go:209] [reset] Deleting content of /var/lib/etcd
[reset] Deleting contents of stateful directories: [/var/lib/etcd /var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
... skipping 21 lines ...
I0320 13:20:53.092947    6070 reset.go:99] [reset] Could not obtain a client set from the kubeconfig file: /etc/kubernetes/admin.conf
I0320 13:20:53.093014    6070 reset.go:118] [reset] Detected and using CRI socket: /var/run/dockershim.sock
I0320 13:20:53.093057    6070 removeetcdmember.go:54] [reset] Checking for etcd config
W0320 13:20:53.093062    6070 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
I0320 13:20:53.093090    6070 cleanupnode.go:57] [reset] Getting init system
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0320 13:20:53.202637    6070 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0320 13:20:53.293356    6070 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
I0320 13:20:59.421715    6070 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
I0320 13:20:59.422099    6070 reset.go:209] [reset] Deleting content of /var/lib/kubelet
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
... skipping 20 lines ...
I0320 13:21:00.427137    5773 reset.go:99] [reset] Could not obtain a client set from the kubeconfig file: /etc/kubernetes/admin.conf
I0320 13:21:00.427203    5773 reset.go:118] [reset] Detected and using CRI socket: /var/run/dockershim.sock
I0320 13:21:00.427245    5773 removeetcdmember.go:54] [reset] Checking for etcd config
W0320 13:21:00.427250    5773 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
I0320 13:21:00.427274    5773 cleanupnode.go:57] [reset] Getting init system
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0320 13:21:00.517902    5773 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0320 13:21:00.590283    5773 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
I0320 13:21:06.889847    5773 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
I0320 13:21:06.890238    5773 reset.go:209] [reset] Deleting content of /var/lib/kubelet
... skipping 19 lines ...
Deleting cluster "kinder-upgrade" ...
time="13:21:07" level=debug msg="Running: /usr/bin/docker [docker ps -q -a --no-trunc --filter label=io.k8s.sigs.kind.cluster --format {{.Names}}\\t{{.Label \"io.k8s.sigs.kind.cluster\"}} --filter label=io.k8s.sigs.kind.cluster=kinder-upgrade]"
time="13:21:07" level=debug msg="Running: /usr/bin/docker [docker rm -f -v kinder-upgrade-lb kinder-upgrade-control-plane-1 kinder-upgrade-worker-1 kinder-upgrade-worker-2 kinder-upgrade-control-plane-3 kinder-upgrade-control-plane-2]"
 completed!

Ran 10 of 13 tasks in 0.000 seconds
FAIL! -- 9 tasks Passed | 1 Failed | 3 Skipped

see junit-runner.xml and task logs files for more details

Error: failed executing the workflow
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up binfmt_misc ...
================================================================================
... skipping 2 lines ...