go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\smultiVolume\s\[Slow\]\sshould\sconcurrently\saccess\sthe\svolume\sand\srestored\ssnapshot\sfrom\spods\son\sthe\ssame\snode\s\[LinuxOnly\]\[Feature\:VolumeSnapshotDataSource\]\[Feature\:VolumeSourceXFS\]$'
test/e2e/storage/testsuites/multivolume.go:323 May 23 06:23:53.378: Unexpected error: <*errors.errorString | 0xc00492b3f0>: { s: "VolumeSnapshot snapshot-9vs7d is not ready within 5m0s", } VolumeSnapshot snapshot-9vs7d is not ready within 5m0s occurred test/e2e/storage/utils/snapshot.go:86from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Creating a kubernetes client May 23 06:18:24.155: INFO: >>> kubeConfig: /workspace/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename multivolume �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] test/e2e/storage/testsuites/multivolume.go:323 �[1mSTEP�[0m: deploying csi gce-pd driver �[1mSTEP�[0m: Building a driver namespace object, basename multivolume-6293 �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace May 23 06:18:24.727: INFO: Found CI service account key at /etc/service-account/service-account.json May 23 06:18:24.727: INFO: Running cp [/etc/service-account/service-account.json /tmp/2172499d-eb94-425a-8066-2a3abd6343a6/cloud-sa.json] May 23 06:18:24.769: INFO: Shredding file /tmp/2172499d-eb94-425a-8066-2a3abd6343a6/cloud-sa.json May 23 06:18:24.769: INFO: Running shred [--remove /tmp/2172499d-eb94-425a-8066-2a3abd6343a6/cloud-sa.json] May 23 06:18:24.801: INFO: File /tmp/2172499d-eb94-425a-8066-2a3abd6343a6/cloud-sa.json successfully shredded May 23 06:18:24.808: INFO: creating *v1.ServiceAccount: multivolume-6293-620/csi-attacher May 23 06:18:24.847: INFO: creating *v1.ClusterRole: external-attacher-runner-multivolume-6293 May 23 06:18:24.847: INFO: Define cluster role external-attacher-runner-multivolume-6293 May 23 06:18:24.886: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-multivolume-6293 May 23 06:18:24.925: INFO: creating *v1.Role: multivolume-6293-620/external-attacher-cfg-multivolume-6293 May 23 06:18:24.965: INFO: creating *v1.RoleBinding: multivolume-6293-620/csi-attacher-role-cfg May 23 06:18:25.004: INFO: creating *v1.ServiceAccount: multivolume-6293-620/csi-provisioner May 23 06:18:25.044: INFO: creating *v1.ClusterRole: external-provisioner-runner-multivolume-6293 May 23 06:18:25.044: INFO: Define cluster role external-provisioner-runner-multivolume-6293 May 23 06:18:25.085: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-multivolume-6293 May 23 06:18:25.124: INFO: creating *v1.Role: multivolume-6293-620/external-provisioner-cfg-multivolume-6293 May 23 06:18:25.164: INFO: creating *v1.RoleBinding: multivolume-6293-620/csi-provisioner-role-cfg May 23 06:18:25.203: INFO: creating *v1.ServiceAccount: multivolume-6293-620/csi-gce-pd-controller-sa May 23 06:18:25.247: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-multivolume-6293 May 23 06:18:25.247: INFO: Define cluster role csi-gce-pd-provisioner-role-multivolume-6293 May 23 06:18:25.286: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-multivolume-6293 May 23 06:18:25.326: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-multivolume-6293 May 23 06:18:25.326: INFO: Define cluster role csi-gce-pd-attacher-role-multivolume-6293 May 23 06:18:25.365: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-multivolume-6293 May 23 06:18:25.404: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-multivolume-6293 May 23 06:18:25.404: INFO: Define cluster role csi-gce-pd-resizer-role-multivolume-6293 May 23 06:18:25.445: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-multivolume-6293 May 23 06:18:25.484: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-multivolume-6293 May 23 06:18:25.485: INFO: Define cluster role csi-gce-pd-snapshotter-role-multivolume-6293 May 23 06:18:25.525: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-multivolume-6293 May 23 06:18:25.565: INFO: creating *v1.Role: multivolume-6293-620/csi-gce-pd-leaderelection-role-multivolume-6293 May 23 06:18:25.609: INFO: creating *v1.RoleBinding: multivolume-6293-620/csi-gce-pd-controller-leaderelection-binding May 23 06:18:25.652: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-multivolume-6293 May 23 06:18:25.692: INFO: creating *v1.DaemonSet: multivolume-6293-620/csi-gce-pd-node May 23 06:18:25.737: INFO: creating *v1.StatefulSet: multivolume-6293-620/csi-gce-pd-controller May 23 06:18:25.897: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-qjt1 May 23 06:18:31.077: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-tls5 May 23 06:18:31.116: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-vtst May 23 06:18:31.154: INFO: Creating resource for dynamic PV May 23 06:18:31.154: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(pd.csi.storage.gke.io) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass multivolume-6293cqh5v �[1mSTEP�[0m: creating a claim �[1mSTEP�[0m: Storage class multivolume-6293cqh5v is already created, skipping creation. �[1mSTEP�[0m: Skipping creation of PVC, it already exists �[1mSTEP�[0m: starting gcepd-injector May 23 06:18:31.353: INFO: Waiting up to 5m0s for pod "gcepd-injector" in namespace "multivolume-6293" to be "running" May 23 06:18:31.392: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 39.431296ms May 23 06:18:33.432: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2.079554062s May 23 06:18:35.437: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4.084102575s May 23 06:18:37.432: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 6.07904401s May 23 06:18:39.432: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 8.079078285s May 23 06:18:41.431: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 10.078347358s May 23 06:18:43.431: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 12.078303046s May 23 06:18:45.432: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 14.078649396s May 23 06:18:47.433: INFO: Pod "gcepd-injector": Phase="Running", Reason="", readiness=true. Elapsed: 16.079824284s May 23 06:18:47.433: INFO: Pod "gcepd-injector" satisfied condition "running" �[1mSTEP�[0m: Writing text file contents in the container. May 23 06:18:47.433: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.84.103 --kubeconfig=/workspace/.kube/config --namespace=multivolume-6293 exec gcepd-injector --namespace=multivolume-6293 -- /bin/sh -c echo 'volume content 1653286711274730006' > /opt/0' May 23 06:18:48.063: INFO: stderr: "" May 23 06:18:48.063: INFO: stdout: "" �[1mSTEP�[0m: Checking that text file contents are perfect. May 23 06:18:48.063: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.84.103 --kubeconfig=/workspace/.kube/config --namespace=multivolume-6293 exec gcepd-injector --namespace=multivolume-6293 -- head -c 34 /opt/0' May 23 06:18:48.531: INFO: stderr: "" May 23 06:18:48.531: INFO: stdout: "volume content 1653286711274730006" May 23 06:18:48.531: INFO: ExecWithOptions {Command:[/bin/sh -c test -b /opt/0] Namespace:multivolume-6293 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} May 23 06:18:48.531: INFO: >>> kubeConfig: /workspace/.kube/config May 23 06:18:48.532: INFO: ExecWithOptions: Clientset creation May 23 06:18:48.532: INFO: ExecWithOptions: execute(POST https://34.145.84.103/api/v1/namespaces/multivolume-6293/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-b+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true) May 23 06:18:48.821: INFO: ExecWithOptions {Command:[/bin/sh -c test -d /opt/0] Namespace:multivolume-6293 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} May 23 06:18:48.821: INFO: >>> kubeConfig: /workspace/.kube/config May 23 06:18:48.822: INFO: ExecWithOptions: Clientset creation May 23 06:18:48.822: INFO: ExecWithOptions: execute(POST https://34.145.84.103/api/v1/namespaces/multivolume-6293/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-d+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true) �[1mSTEP�[0m: Deleting pod gcepd-injector in namespace multivolume-6293 May 23 06:18:49.173: INFO: Waiting for pod gcepd-injector to disappear May 23 06:18:49.212: INFO: Pod gcepd-injector still exists May 23 06:18:51.213: INFO: Waiting for pod gcepd-injector to disappear May 23 06:18:51.252: INFO: Pod gcepd-injector no longer exists �[1mSTEP�[0m: creating a SnapshotClass �[1mSTEP�[0m: creating a dynamic VolumeSnapshot May 23 06:18:51.376: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-9vs7d to become ready May 23 06:18:51.424: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:18:53.464: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:18:55.504: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:18:57.544: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:18:59.584: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:01.624: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:03.663: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:05.704: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:07.745: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:09.785: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:11.825: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:13.865: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:15.905: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:17.944: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:19.984: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:22.029: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:24.070: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:26.110: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:28.150: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:30.190: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:32.230: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:34.269: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:36.310: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:38.370: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:40.409: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:42.449: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:44.489: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:46.528: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:48.569: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:50.608: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:52.649: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:54.689: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:56.728: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:19:58.768: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:00.809: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:02.849: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:04.891: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:06.930: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:08.969: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:11.009: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:13.049: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:15.089: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:17.129: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:19.168: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:21.220: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:23.259: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:25.299: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:27.340: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:29.380: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:31.419: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:33.459: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:35.499: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:37.539: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:39.580: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:41.619: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:43.661: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:45.701: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:47.742: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:49.782: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:51.822: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:53.861: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:55.903: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:57.943: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:20:59.982: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:02.022: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:04.062: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:06.102: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:08.142: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:10.181: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:12.222: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:14.261: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:16.301: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:18.341: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:20.381: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:22.420: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:24.459: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:26.501: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:28.541: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:30.584: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:32.627: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:34.672: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:36.712: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:38.751: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:40.796: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:42.835: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:44.874: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:46.913: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:48.953: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:50.995: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:53.037: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:55.081: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:57.121: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:21:59.161: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:01.201: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:03.241: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:05.281: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:07.321: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:09.361: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:11.400: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:13.439: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:15.478: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:17.517: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:19.557: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:21.597: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:23.643: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:25.684: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:27.725: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:29.769: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:31.809: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:33.849: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:35.890: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:37.929: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:39.972: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:42.011: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:44.051: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:46.091: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:48.131: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:50.171: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:52.212: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:54.252: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:56.292: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:22:58.331: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:00.372: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:02.412: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:04.452: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:06.492: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:08.533: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:10.574: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:12.613: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:14.654: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:16.693: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:18.733: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:20.773: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:22.814: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:24.854: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:26.895: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:28.934: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:30.974: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:33.014: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:35.053: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:37.095: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:39.134: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:41.176: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:43.215: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:45.256: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:47.296: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:49.337: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:51.376: INFO: VolumeSnapshot snapshot-9vs7d found but is not ready. May 23 06:23:53.378: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 06:23:53.378: FAIL: Unexpected error: <*errors.errorString | 0xc00492b3f0>: { s: "VolumeSnapshot snapshot-9vs7d is not ready within 5m0s", } VolumeSnapshot snapshot-9vs7d is not ready within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eae40, 0xc0042f4a58?}, 0xc0012f8b08) test/e2e/storage/utils/snapshot.go:86 +0x1ad k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f09442f6580, 0xc00118aea0}, 0xc004b477a0, {{0x718e5df, 0x1a}, {0x0, 0x0}, {0x7133cd2, 0x9}, {0x0, ...}, ...}, ...) test/e2e/storage/framework/snapshot_resource.go:92 +0x246 k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning(_, {{0xc00442ba80, 0x10}, {0x712a743, 0x5}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...) test/e2e/storage/testsuites/provisioning.go:1083 +0x41f k8s.io/kubernetes/test/e2e/storage/testsuites.(*multiVolumeTestSuite).DefineTests.func8() test/e2e/storage/testsuites/multivolume.go:348 +0x613 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: Creating pod1 with a volume on {Name: Selector:map[] Affinity:nil} May 23 06:23:53.464: INFO: Waiting up to 5m0s for pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4" in namespace "multivolume-6293" to be "running" May 23 06:23:53.518: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 53.939522ms May 23 06:23:55.558: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.093661652s May 23 06:23:57.558: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.093131767s May 23 06:23:59.558: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.093753947s May 23 06:24:01.558: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.093249089s May 23 06:24:03.558: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.093492289s May 23 06:24:05.559: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 12.094338204s May 23 06:24:07.557: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 14.09289792s May 23 06:24:09.558: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 16.093082587s May 23 06:24:11.558: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 18.09345494s May 23 06:24:13.559: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Pending", Reason="", readiness=false. Elapsed: 20.094481739s May 23 06:24:15.557: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4": Phase="Running", Reason="", readiness=true. Elapsed: 22.092836831s May 23 06:24:15.557: INFO: Pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4" satisfied condition "running" �[1mSTEP�[0m: Creating pod2 with a volume on {Name: Selector:map[] Affinity:&Affinity{NodeAffinity:&NodeAffinity{RequiredDuringSchedulingIgnoredDuringExecution:&NodeSelector{NodeSelectorTerms:[]NodeSelectorTerm{NodeSelectorTerm{MatchExpressions:[]NodeSelectorRequirement{},MatchFields:[]NodeSelectorRequirement{NodeSelectorRequirement{Key:metadata.name,Operator:In,Values:[bootstrap-e2e-minion-group-qjt1],},},},},},PreferredDuringSchedulingIgnoredDuringExecution:[]PreferredSchedulingTerm{},},PodAffinity:nil,PodAntiAffinity:nil,}} May 23 06:24:15.639: INFO: Waiting up to 5m0s for pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c" in namespace "multivolume-6293" to be "running" May 23 06:24:15.684: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 44.608842ms May 23 06:24:17.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.084095242s May 23 06:24:19.722: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.08318308s May 23 06:24:21.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.084316427s May 23 06:24:23.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.083686576s May 23 06:24:25.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.085181595s May 23 06:24:27.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 12.085065135s May 23 06:24:29.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 14.084403225s May 23 06:24:31.722: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 16.083402089s May 23 06:24:33.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 18.083578901s May 23 06:24:35.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 20.085413913s May 23 06:24:37.725: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 22.085510205s May 23 06:24:39.725: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 24.085591297s May 23 06:24:41.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 26.083865775s May 23 06:24:43.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 28.084971248s May 23 06:24:45.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 30.084615878s May 23 06:24:47.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 32.084255582s May 23 06:24:49.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 34.08378922s May 23 06:24:51.722: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 36.083097095s May 23 06:24:53.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 38.083577502s May 23 06:24:55.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 40.0844471s May 23 06:24:57.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 42.084403486s May 23 06:24:59.722: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 44.083303312s May 23 06:25:01.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 46.084050759s May 23 06:25:03.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 48.084793442s May 23 06:25:05.724: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 50.084505002s May 23 06:25:07.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 52.084264563s May 23 06:25:09.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 54.114624209s May 23 06:25:11.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 56.114952142s May 23 06:25:13.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 58.114388203s May 23 06:25:15.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.115231924s May 23 06:25:17.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.114749536s May 23 06:25:19.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.115011818s May 23 06:25:21.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.114528515s May 23 06:25:23.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.114628698s May 23 06:25:25.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.115238218s May 23 06:25:27.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.114811507s May 23 06:25:29.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.114175482s May 23 06:25:31.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.115052318s May 23 06:25:33.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.114226999s May 23 06:25:35.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.116204665s May 23 06:25:37.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.115124826s May 23 06:25:39.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.115353085s May 23 06:25:41.722: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.083343564s May 23 06:25:43.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.114561912s May 23 06:25:45.759: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.120029711s May 23 06:25:47.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.114749508s May 23 06:25:49.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.114904165s May 23 06:25:51.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.114405133s May 23 06:25:53.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.115084225s May 23 06:25:55.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.115544926s May 23 06:25:57.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.11539193s May 23 06:25:59.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.114390236s May 23 06:26:01.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.114778162s May 23 06:26:03.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.114789572s May 23 06:26:05.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.115054172s May 23 06:26:07.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.11498368s May 23 06:26:09.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.114875195s May 23 06:26:11.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.114096705s May 23 06:26:13.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.114956749s May 23 06:26:15.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.115100245s May 23 06:26:17.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.11430415s May 23 06:26:19.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.115202082s May 23 06:26:21.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.115067627s May 23 06:26:23.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.114472769s May 23 06:26:25.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.115461346s May 23 06:26:27.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.115943806s May 23 06:26:29.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.114721272s May 23 06:26:31.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.114309932s May 23 06:26:33.760: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.12130682s May 23 06:26:35.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.083736899s May 23 06:26:37.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.114465884s May 23 06:26:39.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.114492907s May 23 06:26:41.757: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.117759231s May 23 06:26:43.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.114262484s May 23 06:26:45.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.115807445s May 23 06:26:47.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.115012887s May 23 06:26:49.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.114776412s May 23 06:26:51.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.114481716s May 23 06:26:53.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.115018063s May 23 06:26:55.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.114198941s May 23 06:26:57.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.114599746s May 23 06:26:59.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.114650796s May 23 06:27:01.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.114496133s May 23 06:27:03.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.114923329s May 23 06:27:05.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.114915722s May 23 06:27:07.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.115366693s May 23 06:27:09.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.114541775s May 23 06:27:11.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.115993289s May 23 06:27:13.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.114788854s May 23 06:27:15.723: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.084346131s May 23 06:27:17.764: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.125376212s May 23 06:27:19.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.114867255s May 23 06:27:21.760: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.120488152s May 23 06:27:23.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.11493067s May 23 06:27:25.731: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.091471176s May 23 06:27:27.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.114893445s May 23 06:27:29.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.113926138s May 23 06:27:31.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.114773362s May 23 06:27:33.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.115175125s May 23 06:27:35.815: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.175931413s May 23 06:27:37.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.114694665s May 23 06:27:39.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.11499716s May 23 06:27:41.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.115317275s May 23 06:27:43.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.115042522s May 23 06:27:45.756: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.117043888s May 23 06:27:47.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.115393335s May 23 06:27:49.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.115009857s May 23 06:27:51.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.114884184s May 23 06:27:53.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.114158277s May 23 06:27:55.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.115155067s May 23 06:27:57.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.114466012s May 23 06:27:59.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.114794966s May 23 06:28:01.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.115170908s May 23 06:28:03.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.114923015s May 23 06:28:05.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.115321733s May 23 06:28:07.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.114097407s May 23 06:28:09.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.114754175s May 23 06:28:11.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.114381558s May 23 06:28:13.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.114093384s May 23 06:28:15.761: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.121999824s May 23 06:28:17.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.114668693s May 23 06:28:19.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.114946695s May 23 06:28:21.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.115086802s May 23 06:28:23.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.115662543s May 23 06:28:25.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.115009288s May 23 06:28:27.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.114754637s May 23 06:28:29.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.115563278s May 23 06:28:31.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.114738637s May 23 06:28:33.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.11390678s May 23 06:28:35.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.114844671s May 23 06:28:37.757: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.118218563s May 23 06:28:39.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.114145709s May 23 06:28:41.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.11473611s May 23 06:28:43.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.114985157s May 23 06:28:45.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.115678258s May 23 06:28:47.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.114841921s May 23 06:28:49.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.114690842s May 23 06:28:51.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.114667061s May 23 06:28:53.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.114732975s May 23 06:28:55.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.115049806s May 23 06:28:57.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.114470434s May 23 06:28:59.768: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.128477669s May 23 06:29:01.758: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.119210406s May 23 06:29:03.760: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.120981077s May 23 06:29:05.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.115676098s May 23 06:29:07.755: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.115742795s May 23 06:29:09.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.11470084s May 23 06:29:11.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.114334403s May 23 06:29:13.754: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.114826587s May 23 06:29:15.753: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.113933226s May 23 06:29:15.855: INFO: Pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.215459451s May 23 06:29:15.856: INFO: Timed out while waiting for pod multivolume-6293/pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c to be running. Last observed as: <*v1.Pod | 0xc002809c00>: { TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c", GenerateName: "", Namespace: "multivolume-6293", SelfLink: "", UID: "f060427a-19a1-4e9f-8444-84ba8a8300f8", ResourceVersion: "8362", Generation: 0, CreationTimestamp: { Time: 2022-05-23T06:24:15Z, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: 2022-05-23T06:24:15Z, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:affinity\":{\".\":{},\"f:nodeAffinity\":{\".\":{},\"f:requiredDuringSchedulingIgnoredDuringExecution\":{}}},\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeDevices\":{\".\":{},\"k:{\\\"devicePath\\\":\\\"/mnt/volume1\\\"}\":{\".\":{},\"f:devicePath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{},\"f:seLinuxOptions\":{\".\":{},\"f:level\":{}}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, ], }, Spec: { Volumes: [ { Name: "volume1", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "pd.csi.storage.gke.iocn7t9-restored", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-c66js", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: nil, DownwardAPI: nil, ConfigMap: nil, ServiceAccountToken: { Audience: "", ExpirationSeconds: 3607, Path: "token", }, }, { Secret: nil, DownwardAPI: nil, ConfigMap: { LocalObjectReference: {Name: ...}, Items: [...], Optional: nil, }, ServiceAccountToken: nil, }, { Secret: nil, DownwardAPI: {Items: [...]}, ConfigMap: nil, ServiceAccountToken: nil, }, ], DefaultMode: 420, }, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, ], InitContainers: nil, Containers: [ { Name: "write-pod", Image: "k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5", Command: [ "/bin/sh", "-c", "trap exit TERM; while true; do sleep 1; done", ], Args: nil, WorkingDir: "", Ports: nil, EnvFrom: nil, Env: nil, Resources: {Limits: nil, Requests: nil}, VolumeMounts: [ { Name: "kube-api-access-c66js", ReadOnly: true, MountPath: "/var/run/secrets/kubernetes.io/serviceaccount", SubPath: "", MountPropagation: nil, SubPathExpr: "", }, ], VolumeDevices: [ { Name: "volume1", DevicePath: "/mnt/volume1", }, ], LivenessProbe: nil, ReadinessProbe: nil, StartupProbe: nil, Lifecycle: nil, TerminationMessagePath: "/dev/termination-log", TerminationMessagePolicy: "File", ImagePullPolicy: "IfNotPresent", SecurityContext: { Capabilities: nil, Privileged: false, SELinuxOptions: nil, WindowsOptions: nil, RunAsUser: nil, RunAsGroup: nil, RunAsNonRoot: nil, ReadOnlyRootFilesystem: nil, AllowPrivilegeEscalation: nil, ProcMount: nil, SeccompProfile: nil, }, Stdin: false, StdinOnce: false, TTY: false, }, ], EphemeralContainers: nil, RestartPolicy: "OnFailure", TerminationGracePeriodSeconds: 30, ActiveDeadlineSeconds: nil, DNSPolicy: "ClusterFirst", NodeSelector: nil, ServiceAccountName: "default", DeprecatedServiceAccount: "default", AutomountServiceAccountToken: nil, NodeName: "", HostNetwork: false, HostPID: false, HostIPC: false, ShareProcessNamespace: nil, SecurityContext: { SELinuxOptions: {User: "", Role: "", Type: "", Level: "s0:c0,c1"}, WindowsOptions: nil, RunAsUser: nil, RunAsGroup: nil, RunAsNonRoot: nil, SupplementalGroups: nil, FSGroup: 1000, Sysctls: nil, FSGroupChangePolicy: nil, SeccompProfile: nil, }, ImagePullSecrets: nil, Hostname: "", Subdomain: "", Affinity: { NodeAffinity: { RequiredDuringSchedulingIgnoredDuringExecution: { NodeSelectorTerms: [ { MatchExpressions: nil, MatchFields: [ {Key: "metadata.name", Operator: "In", Values: [...]}, ], }, ], }, PreferredDuringSchedulingIgnoredDuringExecution: nil, }, PodAffinity: nil, PodAntiAffinity: nil, }, SchedulerName: "default-scheduler", Tolerations: [ { Key: "node.kubernetes.io/not-ready", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, { Key: "node.kubernetes.io/unreachable", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, ], HostAliases: nil, PriorityClassName: "", Priority: 0, DNSConfig: nil, ReadinessGates: nil, RuntimeClassName: nil, EnableServiceLinks: true, PreemptionPolicy: "PreemptLowerPriority", Overhead: nil, TopologySpreadConstraints: nil, SetHostnameAsFQDN: nil, OS: nil, }, Status: { Phase: "Pending", Conditions: nil, Message: "", Reason: "", NominatedNodeName: "", HostIP: "", PodIP: "", PodIPs: nil, StartTime: nil, InitContainerStatuses: nil, ContainerStatuses: nil, QOSClass: "BestEffort", EphemeralContainerStatuses: nil, }, } May 23 06:29:15.857: FAIL: Unexpected error: <*errors.errorString | 0xc001ba6710>: { s: "pod \"pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c\" is not Running: timed out while waiting for pod multivolume-6293/pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c to be running", } pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c" is not Running: timed out while waiting for pod multivolume-6293/pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c to be running occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.TestConcurrentAccessToRelatedVolumes(0xc000e07600, {0x7a45528?, 0xc003867380}, {0xc00442ba80, 0x10}, {{0x0, 0x0}, 0x0, 0xc00178c030}, {0xc00492b7f0, ...}, ...) test/e2e/storage/testsuites/multivolume.go:696 +0x418 k8s.io/kubernetes/test/e2e/storage/testsuites.(*multiVolumeTestSuite).DefineTests.func8() test/e2e/storage/testsuites/multivolume.go:370 +0x8f3 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f May 23 06:29:15.857: INFO: Deleting pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c" in namespace "multivolume-6293" May 23 06:29:15.987: INFO: Wait up to 5m0s for pod "pod-42f924b8-e8e5-43c6-a3eb-27553cc1c09c" to be fully deleted May 23 06:29:16.061: INFO: Deleting pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4" in namespace "multivolume-6293" May 23 06:29:16.159: INFO: Wait up to 5m0s for pod "pod-f89bc226-8920-4cc0-afcc-fb5281c6e8e4" to be fully deleted May 23 06:29:20.458: INFO: deleting initClaim "multivolume-6293"/"pd.csi.storage.gke.iocn7t9" May 23 06:29:20.556: INFO: deleting snapshot "multivolume-6293"/"snapshot-9vs7d" May 23 06:29:20.596: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd readyToUse:false] May 23 06:29:20.596: INFO: snapshotContentName snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd May 23 06:29:20.676: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd to be deleted May 23 06:29:20.723: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:22.764: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:24.811: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:26.850: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:28.890: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:30.936: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:32.976: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:35.018: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:37.062: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:39.104: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:41.151: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:43.192: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:45.235: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:47.280: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:49.328: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:51.368: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:53.416: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:55.464: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:57.508: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:29:59.555: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:01.606: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:03.645: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:05.688: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:07.733: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:09.775: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:11.820: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:13.860: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:15.899: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:17.940: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:19.984: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:22.028: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:24.067: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:26.108: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:28.148: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:30.188: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:32.250: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:34.291: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:36.332: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:38.374: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:40.413: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:42.460: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:44.499: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:46.541: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:48.582: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:50.626: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:52.667: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:54.722: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:56.762: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:30:58.802: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:00.844: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:02.887: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:04.928: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:06.968: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:09.011: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:11.050: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:13.096: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:15.136: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:17.177: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:19.217: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:21.257: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:23.297: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:25.338: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:27.379: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:29.419: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:31.461: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:33.501: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:35.542: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:37.583: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:39.624: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:41.664: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:43.706: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:45.747: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:47.789: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:49.829: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:51.869: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:53.909: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:55.950: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:31:57.990: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:00.035: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:02.075: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:04.115: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:06.158: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:08.198: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:10.238: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:12.279: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:14.319: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:16.359: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:18.399: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:20.438: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:22.477: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:24.518: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:26.558: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:28.597: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:30.639: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:32.679: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:34.719: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:36.759: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:38.798: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:40.843: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:42.883: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:44.923: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:46.962: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:49.002: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:51.042: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:53.082: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:55.121: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:57.161: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:32:59.202: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:01.242: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:03.283: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:05.323: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:07.363: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:09.403: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:11.443: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:13.484: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:15.523: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:17.563: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:19.606: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:21.646: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:23.687: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:25.727: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:27.767: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:29.821: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:31.861: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:33.900: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:35.944: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:38.078: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:40.118: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:42.159: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:44.199: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:46.240: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:48.280: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:50.319: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:52.359: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:54.399: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:56.439: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:33:58.479: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:00.520: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:02.560: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:04.601: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:06.642: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:08.682: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:10.724: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:12.764: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:14.807: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:16.847: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:18.887: INFO: volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd has been found and is not deleted May 23 06:34:20.888: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 06:34:20.889: FAIL: Unexpected error: <*errors.errorString | 0xc0023758f0>: { s: "volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd is not deleted within 5m0s", } volumesnapshotcontents snapcontent-9688aeff-9cb1-4933-bcaf-8187c68db5dd is not deleted within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/framework.(*SnapshotResource).CleanupResource(0xc003b76210, 0xc000ebcf80) test/e2e/storage/framework/snapshot_resource.go:204 +0xefd k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning.func1() test/e2e/storage/testsuites/provisioning.go:1098 +0x294 panic({0x6c82880, 0xc001dcc080}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x6309f00, 0x79de820}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc0018e8960, 0x1c3}, {0xc002a3a968?, 0x7126f4b?, 0xc002a3a988?}) test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc00423a8c0, 0x1ae}, {0xc002e931a0?, 0xc00423a8c0?, 0xc002a22000?}) test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc002a3aad0, {0x7a02ab8, 0xa670820}, 0x0, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc002a3aad0, {0x7a02ab8, 0xa670820}, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0xc003aa5000?, {0x79e2440?, 0xc001ba6710?}, {0x0?, 0x2?, 0xc00025e280?}) test/e2e/framework/expect.go:46 +0x9d k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...) test/e2e/framework/expect.go:40 k8s.io/kubernetes/test/e2e/storage/testsuites.TestConcurrentAccessToRelatedVolumes(0xc000e07600, {0x7a45528?, 0xc003867380}, {0xc00442ba80, 0x10}, {{0x0, 0x0}, 0x0, 0xc00178c030}, {0xc00492b7f0, ...}, ...) test/e2e/storage/testsuites/multivolume.go:696 +0x418 k8s.io/kubernetes/test/e2e/storage/testsuites.(*multiVolumeTestSuite).DefineTests.func8() test/e2e/storage/testsuites/multivolume.go:370 +0x8f3 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: Deleting pvc May 23 06:34:20.954: INFO: Deleting PersistentVolumeClaim "pd.csi.storage.gke.iocn7t9" �[1mSTEP�[0m: Deleting sc �[1mSTEP�[0m: deleting the test namespace: multivolume-6293 �[1mSTEP�[0m: Waiting for namespaces [multivolume-6293] to vanish May 23 06:39:21.455: INFO: error deleting namespace multivolume-6293: timed out waiting for the condition �[1mSTEP�[0m: uninstalling csi gce-pd driver May 23 06:39:21.455: INFO: deleting *v1.ServiceAccount: multivolume-6293-620/csi-attacher May 23 06:39:21.559: INFO: deleting *v1.ClusterRole: external-attacher-runner-multivolume-6293 May 23 06:39:21.657: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-multivolume-6293 May 23 06:39:21.755: INFO: deleting *v1.Role: multivolume-6293-620/external-attacher-cfg-multivolume-6293 May 23 06:39:21.855: INFO: deleting *v1.RoleBinding: multivolume-6293-620/csi-attacher-role-cfg May 23 06:39:21.955: INFO: deleting *v1.ServiceAccount: multivolume-6293-620/csi-provisioner May 23 06:39:22.056: INFO: deleting *v1.ClusterRole: external-provisioner-runner-multivolume-6293 May 23 06:39:22.156: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-multivolume-6293 May 23 06:39:22.257: INFO: deleting *v1.Role: multivolume-6293-620/external-provisioner-cfg-multivolume-6293 May 23 06:39:22.358: INFO: deleting *v1.RoleBinding: multivolume-6293-620/csi-provisioner-role-cfg May 23 06:39:22.458: INFO: deleting *v1.ServiceAccount: multivolume-6293-620/csi-gce-pd-controller-sa May 23 06:39:22.555: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-multivolume-6293 May 23 06:39:22.657: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-multivolume-6293 May 23 06:39:22.756: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-multivolume-6293 May 23 06:39:22.876: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-multivolume-6293 May 23 06:39:22.958: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-multivolume-6293 May 23 06:39:23.055: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-multivolume-6293 May 23 06:39:23.156: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-multivolume-6293 May 23 06:39:23.255: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-multivolume-6293 May 23 06:39:23.355: INFO: deleting *v1.Role: multivolume-6293-620/csi-gce-pd-leaderelection-role-multivolume-6293 May 23 06:39:23.456: INFO: deleting *v1.RoleBinding: multivolume-6293-620/csi-gce-pd-controller-leaderelection-binding May 23 06:39:23.556: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-multivolume-6293 May 23 06:39:23.655: INFO: deleting *v1.DaemonSet: multivolume-6293-620/csi-gce-pd-node May 23 06:39:23.756: INFO: deleting *v1.StatefulSet: multivolume-6293-620/csi-gce-pd-controller �[1mSTEP�[0m: deleting the driver namespace: multivolume-6293-620 �[1mSTEP�[0m: Waiting for namespaces [multivolume-6293-620] to vanish [AfterEach] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] test/e2e/framework/framework.go:187 �[1mSTEP�[0m: Collecting events from namespace "multivolume-6293". �[1mSTEP�[0m: Found 0 events. May 23 06:39:30.147: INFO: POD NODE PHASE GRACE CONDITIONS May 23 06:39:30.147: INFO: May 23 06:39:30.188: INFO: Logging node info for node bootstrap-e2e-master May 23 06:39:30.228: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 10972 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:18:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:47 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:47 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:47 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:38:47 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:39:30.228: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 06:39:30.269: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 06:39:30.326: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 06:39:30.326: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 06:39:30.326: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container kube-scheduler ready: true, restart count 0 May 23 06:39:30.326: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 06:39:30.326: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container etcd-container ready: true, restart count 0 May 23 06:39:30.326: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 06:39:30.326: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 06:39:30.326: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:39:30.326: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:39:30.326: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container etcd-container ready: true, restart count 0 May 23 06:39:30.326: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.326: INFO: Container kube-apiserver ready: true, restart count 0 May 23 06:39:30.509: INFO: Latency metrics for node bootstrap-e2e-master May 23 06:39:30.509: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 06:39:30.549: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 10981 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 06:24:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:29:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:36:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:36:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:36:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:36:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:34:31 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:34:31 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:34:31 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:34:31 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:39:30.550: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 06:39:30.591: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 06:39:30.669: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.669: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:39:30.669: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 06:39:30.669: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:39:30.669: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:39:30.669: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.669: INFO: Container coredns ready: true, restart count 0 May 23 06:39:30.669: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.669: INFO: Container autoscaler ready: true, restart count 0 May 23 06:39:30.669: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:39:30.669: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:39:30.848: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 06:39:30.848: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 06:39:30.888: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 10983 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:18:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:36:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:36:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:36:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:36:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:30 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:30 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:30 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:38:30 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:39:30.888: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 06:39:30.931: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 06:39:31.016: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 06:39:31.016: INFO: Container coredns ready: true, restart count 0 May 23 06:39:31.016: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 06:39:31.016: INFO: Container metrics-server ready: true, restart count 0 May 23 06:39:31.016: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 06:39:31.016: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:39:31.016: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:39:31.016: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:39:31.016: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:39:31.016: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:39:31.016: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:39:31.016: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:39:31.186: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 06:39:31.186: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 06:39:31.227: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 10970 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:45:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:18:29 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:36:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:36:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:36:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:36:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:36:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:26 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:26 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:38:26 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:38:26 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:39:31.227: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 06:39:31.268: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 06:39:31.328: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:39:31.328: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:39:31.328: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:39:31.328: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:39:31.328: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:39:31.328: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:39:31.328: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 06:39:31.328: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:39:31.328: INFO: Container default-http-backend ready: true, restart count 0 May 23 06:39:31.328: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:39:31.328: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:39:31.490: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst May 23 06:39:31.491: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "multivolume-6293" for this suite.
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sDynamic\sPV\s\(block\svolmode\)\]\sprovisioning\sshould\sprovision\sstorage\swith\ssnapshot\sdata\ssource\s\[Feature\:VolumeSnapshotDataSource\]$'
test/e2e/storage/testsuites/provisioning.go:208 May 23 06:44:58.800: Unexpected error: <*errors.errorString | 0xc0025ebce0>: { s: "VolumeSnapshot snapshot-blffc is not ready within 5m0s", } VolumeSnapshot snapshot-blffc is not ready within 5m0s occurred test/e2e/storage/utils/snapshot.go:86from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Creating a kubernetes client May 23 06:39:31.571: INFO: >>> kubeConfig: /workspace/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] test/e2e/storage/testsuites/provisioning.go:208 �[1mSTEP�[0m: deploying csi gce-pd driver �[1mSTEP�[0m: Building a driver namespace object, basename provisioning-2010 �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace May 23 06:39:32.158: INFO: Found CI service account key at /etc/service-account/service-account.json May 23 06:39:32.158: INFO: Running cp [/etc/service-account/service-account.json /tmp/44d9778e-db32-4509-bb6b-fd6fc5eb5013/cloud-sa.json] May 23 06:39:32.202: INFO: Shredding file /tmp/44d9778e-db32-4509-bb6b-fd6fc5eb5013/cloud-sa.json May 23 06:39:32.202: INFO: Running shred [--remove /tmp/44d9778e-db32-4509-bb6b-fd6fc5eb5013/cloud-sa.json] May 23 06:39:32.236: INFO: File /tmp/44d9778e-db32-4509-bb6b-fd6fc5eb5013/cloud-sa.json successfully shredded May 23 06:39:32.243: INFO: creating *v1.ServiceAccount: provisioning-2010-2442/csi-attacher May 23 06:39:32.285: INFO: creating *v1.ClusterRole: external-attacher-runner-provisioning-2010 May 23 06:39:32.285: INFO: Define cluster role external-attacher-runner-provisioning-2010 May 23 06:39:32.325: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-provisioning-2010 May 23 06:39:32.366: INFO: creating *v1.Role: provisioning-2010-2442/external-attacher-cfg-provisioning-2010 May 23 06:39:32.407: INFO: creating *v1.RoleBinding: provisioning-2010-2442/csi-attacher-role-cfg May 23 06:39:32.449: INFO: creating *v1.ServiceAccount: provisioning-2010-2442/csi-provisioner May 23 06:39:32.492: INFO: creating *v1.ClusterRole: external-provisioner-runner-provisioning-2010 May 23 06:39:32.492: INFO: Define cluster role external-provisioner-runner-provisioning-2010 May 23 06:39:32.534: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-2010 May 23 06:39:32.573: INFO: creating *v1.Role: provisioning-2010-2442/external-provisioner-cfg-provisioning-2010 May 23 06:39:32.614: INFO: creating *v1.RoleBinding: provisioning-2010-2442/csi-provisioner-role-cfg May 23 06:39:32.654: INFO: creating *v1.ServiceAccount: provisioning-2010-2442/csi-gce-pd-controller-sa May 23 06:39:32.694: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-provisioning-2010 May 23 06:39:32.694: INFO: Define cluster role csi-gce-pd-provisioner-role-provisioning-2010 May 23 06:39:32.736: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-provisioning-2010 May 23 06:39:32.791: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-provisioning-2010 May 23 06:39:32.791: INFO: Define cluster role csi-gce-pd-attacher-role-provisioning-2010 May 23 06:39:32.831: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-provisioning-2010 May 23 06:39:32.873: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-provisioning-2010 May 23 06:39:32.873: INFO: Define cluster role csi-gce-pd-resizer-role-provisioning-2010 May 23 06:39:32.922: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-provisioning-2010 May 23 06:39:32.971: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-provisioning-2010 May 23 06:39:32.971: INFO: Define cluster role csi-gce-pd-snapshotter-role-provisioning-2010 May 23 06:39:33.026: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-provisioning-2010 May 23 06:39:33.080: INFO: creating *v1.Role: provisioning-2010-2442/csi-gce-pd-leaderelection-role-provisioning-2010 May 23 06:39:33.158: INFO: creating *v1.RoleBinding: provisioning-2010-2442/csi-gce-pd-controller-leaderelection-binding May 23 06:39:33.224: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-provisioning-2010 May 23 06:39:33.265: INFO: creating *v1.DaemonSet: provisioning-2010-2442/csi-gce-pd-node May 23 06:39:33.308: INFO: creating *v1.StatefulSet: provisioning-2010-2442/csi-gce-pd-controller May 23 06:39:33.432: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-qjt1 May 23 06:39:38.595: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-tls5 May 23 06:39:38.634: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-vtst May 23 06:39:38.672: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-2010pfjmd 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:pd.csi.storage.gke.io,Parameters:map[string]string{type: pd-standard,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-2010 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-2010pfjmd,VolumeMode:*Block,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-2010 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-2010pfjmd,VolumeMode:*Block,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP�[0m: Creating a StorageClass �[1mSTEP�[0m: [Initialize dataSource]creating a initClaim �[1mSTEP�[0m: starting gcepd-injector May 23 06:39:38.872: INFO: Waiting up to 5m0s for pod "gcepd-injector" in namespace "provisioning-2010" to be "running" May 23 06:39:38.911: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 38.605545ms May 23 06:39:40.952: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2.079501975s May 23 06:39:42.950: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4.077159299s May 23 06:39:44.950: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 6.077791422s May 23 06:39:46.950: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 8.077568194s May 23 06:39:48.951: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 10.078039923s May 23 06:39:50.950: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 12.077889741s May 23 06:39:52.951: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 14.078352113s May 23 06:39:54.951: INFO: Pod "gcepd-injector": Phase="Running", Reason="", readiness=true. Elapsed: 16.078885847s May 23 06:39:54.951: INFO: Pod "gcepd-injector" satisfied condition "running" �[1mSTEP�[0m: Writing text file contents in the container. May 23 06:39:54.952: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.84.103 --kubeconfig=/workspace/.kube/config --namespace=provisioning-2010 exec gcepd-injector --namespace=provisioning-2010 -- /bin/sh -c echo 'Hello from namespace provisioning-2010' > /opt/0' May 23 06:39:55.435: INFO: stderr: "" May 23 06:39:55.435: INFO: stdout: "" �[1mSTEP�[0m: Checking that text file contents are perfect. May 23 06:39:55.435: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.84.103 --kubeconfig=/workspace/.kube/config --namespace=provisioning-2010 exec gcepd-injector --namespace=provisioning-2010 -- head -c 38 /opt/0' May 23 06:39:55.946: INFO: stderr: "" May 23 06:39:55.946: INFO: stdout: "Hello from namespace provisioning-2010" May 23 06:39:55.946: INFO: ExecWithOptions {Command:[/bin/sh -c test -b /opt/0] Namespace:provisioning-2010 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} May 23 06:39:55.946: INFO: >>> kubeConfig: /workspace/.kube/config May 23 06:39:55.947: INFO: ExecWithOptions: Clientset creation May 23 06:39:55.947: INFO: ExecWithOptions: execute(POST https://34.145.84.103/api/v1/namespaces/provisioning-2010/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-b+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true) May 23 06:39:56.252: INFO: ExecWithOptions {Command:[/bin/sh -c test -d /opt/0] Namespace:provisioning-2010 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} May 23 06:39:56.252: INFO: >>> kubeConfig: /workspace/.kube/config May 23 06:39:56.253: INFO: ExecWithOptions: Clientset creation May 23 06:39:56.253: INFO: ExecWithOptions: execute(POST https://34.145.84.103/api/v1/namespaces/provisioning-2010/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-d+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true) �[1mSTEP�[0m: Deleting pod gcepd-injector in namespace provisioning-2010 May 23 06:39:56.585: INFO: Waiting for pod gcepd-injector to disappear May 23 06:39:56.623: INFO: Pod gcepd-injector still exists May 23 06:39:58.624: INFO: Waiting for pod gcepd-injector to disappear May 23 06:39:58.662: INFO: Pod gcepd-injector no longer exists �[1mSTEP�[0m: creating a SnapshotClass �[1mSTEP�[0m: creating a dynamic VolumeSnapshot May 23 06:39:58.785: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-blffc to become ready May 23 06:39:58.828: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:00.868: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:02.907: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:04.948: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:06.987: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:09.033: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:11.072: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:13.112: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:15.153: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:17.193: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:19.233: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:21.272: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:23.312: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:25.352: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:27.395: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:29.435: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:31.476: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:33.516: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:35.556: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:37.595: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:39.634: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:41.675: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:43.715: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:45.755: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:47.795: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:49.835: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:51.874: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:53.914: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:55.955: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:40:57.995: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:00.035: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:02.076: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:04.115: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:06.155: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:08.195: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:10.238: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:12.280: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:14.319: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:16.359: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:18.398: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:20.438: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:22.477: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:24.517: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:26.557: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:28.596: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:30.636: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:32.676: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:34.716: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:36.756: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:38.797: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:40.837: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:42.877: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:44.917: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:46.958: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:48.997: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:51.037: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:53.077: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:55.117: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:57.156: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:41:59.196: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:01.236: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:03.276: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:05.317: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:07.358: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:09.398: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:11.437: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:13.477: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:15.518: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:17.559: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:19.599: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:21.639: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:23.682: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:25.721: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:27.761: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:29.802: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:31.841: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:33.881: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:35.922: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:37.962: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:40.003: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:42.045: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:44.123: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:46.169: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:48.207: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:50.247: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:52.289: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:54.329: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:56.368: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:42:58.408: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:00.449: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:02.489: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:04.529: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:06.568: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:08.608: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:10.650: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:12.691: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:14.733: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:16.773: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:18.814: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:20.854: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:22.893: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:24.933: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:26.975: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:29.015: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:31.055: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:33.095: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:35.150: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:37.190: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:39.229: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:41.269: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:43.310: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:45.350: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:47.390: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:49.430: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:51.507: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:53.546: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:55.595: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:57.635: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:43:59.675: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:01.715: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:03.755: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:05.795: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:07.835: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:09.876: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:11.915: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:13.954: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:15.998: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:18.038: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:20.079: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:22.120: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:24.159: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:26.199: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:28.239: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:30.279: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:32.319: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:34.359: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:36.398: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:38.438: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:40.477: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:42.519: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:44.559: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:46.599: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:48.638: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:50.680: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:52.719: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:54.759: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:56.799: INFO: VolumeSnapshot snapshot-blffc found but is not ready. May 23 06:44:58.799: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 06:44:58.800: FAIL: Unexpected error: <*errors.errorString | 0xc0025ebce0>: { s: "VolumeSnapshot snapshot-blffc is not ready within 5m0s", } VolumeSnapshot snapshot-blffc is not ready within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eae40, 0xc003ee5bf8?}, 0xc00084c1b8) test/e2e/storage/utils/snapshot.go:86 +0x1ad k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f09442f6580, 0xc00118aea0}, 0xc004b46c00, {{0x718e5df, 0x1a}, {0x0, 0x0}, {0x7133cd2, 0x9}, {0x0, ...}, ...}, ...) test/e2e/storage/framework/snapshot_resource.go:92 +0x246 k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning(_, {{0xc0048a4588, 0x11}, {0x712a743, 0x5}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...) test/e2e/storage/testsuites/provisioning.go:1083 +0x41f k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4() test/e2e/storage/testsuites/provisioning.go:227 +0x485 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-2010 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-2010pfjmd,VolumeMode:*Block,DataSource:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-blffc,},DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-2010pfjmd ab92d9fa-0c15-4cea-a802-6f3d9624e8f1 11107 0 2022-05-23 06:39:38 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2022-05-23 06:39:38 +0000 UTC FieldsV1 {"f:parameters":{".":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:pd.csi.storage.gke.io,Parameters:map[string]string{type: pd-standard,},ReclaimPolicy:*Delete,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-ndt6b pvc- provisioning-2010 96152bfb-840a-4836-a53a-30ee8b0fb743 12037 0 2022-05-23 06:44:58 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2022-05-23 06:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:dataSource":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-2010pfjmd,VolumeMode:*Block,DataSource:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-blffc,},DataSourceRef:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-blffc,},},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} May 23 06:44:58.926: INFO: Waiting up to 5m0s for pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0" in namespace "provisioning-2010" to be "running" May 23 06:44:58.964: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 38.446491ms May 23 06:45:01.006: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.080204737s May 23 06:45:03.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.078666846s May 23 06:45:05.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 6.077838365s May 23 06:45:07.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 8.079038642s May 23 06:45:09.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 10.078151827s May 23 06:45:11.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 12.078780294s May 23 06:45:13.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 14.077300205s May 23 06:45:15.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 16.0774115s May 23 06:45:17.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 18.078206551s May 23 06:45:19.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 20.07854161s May 23 06:45:21.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 22.077487778s May 23 06:45:23.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 24.077740143s May 23 06:45:25.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 26.077821769s May 23 06:45:27.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 28.079279975s May 23 06:45:29.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 30.077840168s May 23 06:45:31.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 32.078294216s May 23 06:45:33.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 34.079367764s May 23 06:45:35.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 36.079699434s May 23 06:45:37.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 38.07923569s May 23 06:45:39.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 40.077850178s May 23 06:45:41.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 42.078831875s May 23 06:45:43.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 44.077556394s May 23 06:45:45.007: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 46.081021418s May 23 06:45:47.008: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 48.082020138s May 23 06:45:49.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 50.078437184s May 23 06:45:51.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 52.078315732s May 23 06:45:53.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 54.078872464s May 23 06:45:55.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 56.077798455s May 23 06:45:57.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 58.077823517s May 23 06:45:59.007: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.081143778s May 23 06:46:01.007: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.081013083s May 23 06:46:03.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.078072852s May 23 06:46:05.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.077709263s May 23 06:46:07.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.121825241s May 23 06:46:09.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.121954906s May 23 06:46:11.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.122213338s May 23 06:46:13.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.121975536s May 23 06:46:15.049: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.123382976s May 23 06:46:17.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.122147204s May 23 06:46:19.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.122255886s May 23 06:46:21.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.077807274s May 23 06:46:23.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.121703388s May 23 06:46:25.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.121522023s May 23 06:46:27.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.12266842s May 23 06:46:29.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.122541671s May 23 06:46:31.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.121611876s May 23 06:46:33.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.121905382s May 23 06:46:35.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.122044739s May 23 06:46:37.065: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.139024321s May 23 06:46:39.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.122156621s May 23 06:46:41.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.122577419s May 23 06:46:43.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.121998029s May 23 06:46:45.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.121339555s May 23 06:46:47.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.121531283s May 23 06:46:49.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.121391926s May 23 06:46:51.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.122026249s May 23 06:46:53.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.12144964s May 23 06:46:55.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.12208489s May 23 06:46:57.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.121706677s May 23 06:46:59.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.122403523s May 23 06:47:01.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.121840968s May 23 06:47:03.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.122402095s May 23 06:47:05.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.121683005s May 23 06:47:07.049: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.122954847s May 23 06:47:09.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.121827593s May 23 06:47:11.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.078368827s May 23 06:47:13.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.122317465s May 23 06:47:15.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.122043933s May 23 06:47:17.049: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.122855357s May 23 06:47:19.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.122478905s May 23 06:47:21.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.12251521s May 23 06:47:23.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.121916168s May 23 06:47:25.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.121816063s May 23 06:47:27.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.122045209s May 23 06:47:29.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.122382614s May 23 06:47:31.014: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.087942503s May 23 06:47:33.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.122348683s May 23 06:47:35.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.122228087s May 23 06:47:37.073: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.146965403s May 23 06:47:39.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.121642946s May 23 06:47:41.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.122101382s May 23 06:47:43.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.077437779s May 23 06:47:45.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.121374886s May 23 06:47:47.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.122118904s May 23 06:47:49.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.121050342s May 23 06:47:51.050: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.12384345s May 23 06:47:53.016: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.090188844s May 23 06:47:55.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.121249813s May 23 06:47:57.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.121769178s May 23 06:47:59.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.121499774s May 23 06:48:01.062: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.136425423s May 23 06:48:03.051: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.125049365s May 23 06:48:05.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.122238842s May 23 06:48:07.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.12156646s May 23 06:48:09.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.121753437s May 23 06:48:11.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.122649557s May 23 06:48:13.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.077212757s May 23 06:48:15.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.121316359s May 23 06:48:17.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.122052545s May 23 06:48:19.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.121695709s May 23 06:48:21.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.121988907s May 23 06:48:23.074: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.14793579s May 23 06:48:25.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.121587028s May 23 06:48:27.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.121609782s May 23 06:48:29.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.12147263s May 23 06:48:31.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.121987492s May 23 06:48:33.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.121792639s May 23 06:48:35.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.121791362s May 23 06:48:37.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.121262206s May 23 06:48:39.004: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.078140721s May 23 06:48:41.051: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.124976672s May 23 06:48:43.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.121491233s May 23 06:48:45.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.121608093s May 23 06:48:47.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.121411512s May 23 06:48:49.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.121088512s May 23 06:48:51.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.079225909s May 23 06:48:53.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.121448096s May 23 06:48:55.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.121695505s May 23 06:48:57.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.121704802s May 23 06:48:59.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.122138966s May 23 06:49:01.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.12198634s May 23 06:49:03.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.122121436s May 23 06:49:05.005: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.078909812s May 23 06:49:07.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.121348064s May 23 06:49:09.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.12182943s May 23 06:49:11.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.121982489s May 23 06:49:13.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.121913472s May 23 06:49:15.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.121422125s May 23 06:49:17.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.120990487s May 23 06:49:19.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.121562536s May 23 06:49:21.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.121639371s May 23 06:49:23.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.121315566s May 23 06:49:25.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.077438183s May 23 06:49:27.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.122170646s May 23 06:49:29.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.121578727s May 23 06:49:31.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.121563847s May 23 06:49:33.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.122626076s May 23 06:49:35.054: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.127875321s May 23 06:49:37.003: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.077584307s May 23 06:49:39.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.121592916s May 23 06:49:41.049: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.122804559s May 23 06:49:43.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.121998146s May 23 06:49:45.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.122218112s May 23 06:49:47.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.121457166s May 23 06:49:49.048: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.121893889s May 23 06:49:51.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.12146508s May 23 06:49:53.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.121555889s May 23 06:49:55.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.121060536s May 23 06:49:57.059: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.133446578s May 23 06:49:59.047: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.121474365s May 23 06:49:59.148: INFO: Pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.222331946s May 23 06:49:59.149: INFO: Timed out while waiting for pod provisioning-2010/pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0 to be running. Last observed as: <*v1.Pod | 0xc0046a4000>: { TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0", GenerateName: "", Namespace: "provisioning-2010", SelfLink: "", UID: "469b9b6e-0451-42d3-a3c7-5cfbff5042a3", ResourceVersion: "12038", Generation: 0, CreationTimestamp: { Time: 2022-05-23T06:44:58Z, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: 2022-05-23T06:44:58Z, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeDevices\":{\".\":{},\"k:{\\\"devicePath\\\":\\\"/mnt/volume1\\\"}\":{\".\":{},\"f:devicePath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, ], }, Spec: { Volumes: [ { Name: "volume1", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "pvc-ndt6b", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-kpp9r", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: nil, DownwardAPI: nil, ConfigMap: nil, ServiceAccountToken: { Audience: "", ExpirationSeconds: 3607, Path: "token", }, }, { Secret: nil, DownwardAPI: nil, ConfigMap: { LocalObjectReference: {Name: ...}, Items: [...], Optional: nil, }, ServiceAccountToken: nil, }, { Secret: nil, DownwardAPI: {Items: [...]}, ConfigMap: nil, ServiceAccountToken: nil, }, ], DefaultMode: 420, }, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, ], InitContainers: nil, Containers: [ { Name: "write-pod", Image: "k8s.gcr.io/e2e-test-images/busybox:1.29-2", Command: [ "/bin/sh", "-c", "trap exit TERM; while true; do sleep 1; done", ], Args: nil, WorkingDir: "", Ports: nil, EnvFrom: nil, Env: nil, Resources: {Limits: nil, Requests: nil}, VolumeMounts: [ { Name: "kube-api-access-kpp9r", ReadOnly: true, MountPath: "/var/run/secrets/kubernetes.io/serviceaccount", SubPath: "", MountPropagation: nil, SubPathExpr: "", }, ], VolumeDevices: [ { Name: "volume1", DevicePath: "/mnt/volume1", }, ], LivenessProbe: nil, ReadinessProbe: nil, StartupProbe: nil, Lifecycle: nil, TerminationMessagePath: "/dev/termination-log", TerminationMessagePolicy: "File", ImagePullPolicy: "IfNotPresent", SecurityContext: { Capabilities: nil, Privileged: false, SELinuxOptions: nil, WindowsOptions: nil, RunAsUser: nil, RunAsGroup: nil, RunAsNonRoot: nil, ReadOnlyRootFilesystem: nil, AllowPrivilegeEscalation: nil, ProcMount: nil, SeccompProfile: nil, }, Stdin: false, StdinOnce: false, TTY: false, }, ], EphemeralContainers: nil, RestartPolicy: "OnFailure", TerminationGracePeriodSeconds: 30, ActiveDeadlineSeconds: nil, DNSPolicy: "ClusterFirst", NodeSelector: nil, ServiceAccountName: "default", DeprecatedServiceAccount: "default", AutomountServiceAccountToken: nil, NodeName: "", HostNetwork: false, HostPID: false, HostIPC: false, ShareProcessNamespace: nil, SecurityContext: { SELinuxOptions: nil, WindowsOptions: nil, RunAsUser: nil, RunAsGroup: nil, RunAsNonRoot: nil, SupplementalGroups: nil, FSGroup: 1000, Sysctls: nil, FSGroupChangePolicy: nil, SeccompProfile: nil, }, ImagePullSecrets: nil, Hostname: "", Subdomain: "", Affinity: nil, SchedulerName: "default-scheduler", Tolerations: [ { Key: "node.kubernetes.io/not-ready", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, { Key: "node.kubernetes.io/unreachable", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, ], HostAliases: nil, PriorityClassName: "", Priority: 0, DNSConfig: nil, ReadinessGates: nil, RuntimeClassName: nil, EnableServiceLinks: true, PreemptionPolicy: "PreemptLowerPriority", Overhead: nil, TopologySpreadConstraints: nil, SetHostnameAsFQDN: nil, OS: nil, }, Status: { Phase: "Pending", Conditions: nil, Message: "", Reason: "", NominatedNodeName: "", HostIP: "", PodIP: "", PodIPs: nil, StartTime: nil, InitContainerStatuses: nil, ContainerStatuses: nil, QOSClass: "BestEffort", EphemeralContainerStatuses: nil, }, } May 23 06:49:59.149: FAIL: Unexpected error: <*errors.errorString | 0xc003c9c180>: { s: "pod \"pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0\" is not Running: timed out while waiting for pod provisioning-2010/pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0 to be running", } pod "pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0" is not Running: timed out while waiting for pod provisioning-2010/pod-79f67dbb-61ff-4e88-b05a-4b4b465ddeb0 to be running occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning({{0x7a45528, 0xc002eec600}, 0xc000ebc580, 0xc001385880, 0xc001385dc0, 0xc003867500, {0x0, 0x0}, {0x0, 0x0, ...}, ...}) test/e2e/storage/testsuites/provisioning.go:624 +0x808 k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4() test/e2e/storage/testsuites/provisioning.go:243 +0x618 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f May 23 06:49:59.150: INFO: deleting claim "provisioning-2010"/"pvc-ndt6b" May 23 06:49:59.251: INFO: deleting initClaim "provisioning-2010"/"pvc-cz4lm" May 23 06:49:59.349: INFO: deleting snapshot "provisioning-2010"/"snapshot-blffc" May 23 06:49:59.387: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 readyToUse:false] May 23 06:49:59.387: INFO: snapshotContentName snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 May 23 06:49:59.467: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 to be deleted May 23 06:49:59.513: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:01.552: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:03.591: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:05.630: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:07.670: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:09.710: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:11.750: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:13.791: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:15.855: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:17.894: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:19.934: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:21.974: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:24.013: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:26.058: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:28.102: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:30.142: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:32.183: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:34.223: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:36.266: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:38.305: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:40.347: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:42.388: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:44.427: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:46.466: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:48.505: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:50.545: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:52.585: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:54.626: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:56.666: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:50:58.715: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:00.759: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:02.802: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:04.842: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:06.883: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:08.930: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:10.970: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:13.010: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:15.051: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:17.092: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:19.132: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:21.171: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:23.211: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:25.251: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:27.290: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:29.330: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:31.370: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:33.409: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:35.451: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:37.491: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:39.531: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:41.570: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:43.610: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:45.650: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:47.690: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:49.730: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:51.770: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:53.810: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:55.852: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:57.891: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:51:59.931: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:01.989: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:04.034: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:06.074: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:08.113: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:10.152: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:12.193: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:14.240: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:16.282: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:18.321: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:20.361: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:22.402: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:24.443: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:26.484: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:28.524: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:30.565: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:32.606: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:34.646: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:36.685: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:38.725: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:40.765: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:42.805: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:44.844: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:46.884: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:48.925: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:50.969: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:53.009: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:55.051: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:57.091: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:52:59.131: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:01.171: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:03.211: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:05.251: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:07.292: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:09.331: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:11.371: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:13.409: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:15.451: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:17.490: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:19.530: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:21.570: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:23.610: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:25.650: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:27.689: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:29.730: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:31.780: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:33.821: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:35.861: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:37.901: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:39.942: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:41.982: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:44.025: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:46.065: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:48.105: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:50.146: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:52.186: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:54.225: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:56.265: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:53:58.304: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:00.344: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:02.385: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:04.425: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:06.465: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:08.505: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:10.545: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:12.585: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:14.624: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:16.665: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:18.705: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:20.750: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:22.790: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:24.834: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:26.874: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:28.915: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:30.954: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:32.994: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:35.036: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:37.077: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:39.118: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:41.158: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:43.201: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:45.241: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:47.282: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:49.323: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:51.362: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:53.402: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:55.442: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:57.482: INFO: volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 has been found and is not deleted May 23 06:54:59.484: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 06:54:59.485: FAIL: Unexpected error: <*errors.errorString | 0xc003fedef0>: { s: "volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 is not deleted within 5m0s", } volumesnapshotcontents snapcontent-f9bd6d2b-aac4-406c-8712-45595c2ea7d5 is not deleted within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/framework.(*SnapshotResource).CleanupResource(0xc0004f7130, 0xc000ebc580) test/e2e/storage/framework/snapshot_resource.go:204 +0xefd k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning.func1() test/e2e/storage/testsuites/provisioning.go:1098 +0x294 panic({0x6c82880, 0xc00451dbc0}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x6309f00, 0x79de820}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc0018e8960, 0x1c5}, {0xc001964888?, 0x7126f4b?, 0xc0019648a8?}) test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc002983a40, 0x1b0}, {0xc00470c710?, 0xc002983a40?, 0xc0027c4000?}) test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc0019649f0, {0x7a02ab8, 0xa670820}, 0x0, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc0019649f0, {0x7a02ab8, 0xa670820}, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x7a45528?, {0x79e2440?, 0xc003c9c180?}, {0x0?, 0x2?, 0x0?}) test/e2e/framework/expect.go:46 +0x9d k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...) test/e2e/framework/expect.go:40 k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning({{0x7a45528, 0xc002eec600}, 0xc000ebc580, 0xc001385880, 0xc001385dc0, 0xc003867500, {0x0, 0x0}, {0x0, 0x0, ...}, ...}) test/e2e/storage/testsuites/provisioning.go:624 +0x808 k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4() test/e2e/storage/testsuites/provisioning.go:243 +0x618 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: deleting the test namespace: provisioning-2010 �[1mSTEP�[0m: Waiting for namespaces [provisioning-2010] to vanish May 23 06:59:59.749: INFO: error deleting namespace provisioning-2010: timed out waiting for the condition �[1mSTEP�[0m: uninstalling csi gce-pd driver May 23 06:59:59.749: INFO: deleting *v1.ServiceAccount: provisioning-2010-2442/csi-attacher May 23 06:59:59.851: INFO: deleting *v1.ClusterRole: external-attacher-runner-provisioning-2010 May 23 06:59:59.949: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-provisioning-2010 May 23 07:00:00.050: INFO: deleting *v1.Role: provisioning-2010-2442/external-attacher-cfg-provisioning-2010 May 23 07:00:00.154: INFO: deleting *v1.RoleBinding: provisioning-2010-2442/csi-attacher-role-cfg May 23 07:00:00.257: INFO: deleting *v1.ServiceAccount: provisioning-2010-2442/csi-provisioner May 23 07:00:00.350: INFO: deleting *v1.ClusterRole: external-provisioner-runner-provisioning-2010 May 23 07:00:00.451: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-2010 May 23 07:00:00.571: INFO: deleting *v1.Role: provisioning-2010-2442/external-provisioner-cfg-provisioning-2010 May 23 07:00:00.657: INFO: deleting *v1.RoleBinding: provisioning-2010-2442/csi-provisioner-role-cfg May 23 07:00:00.759: INFO: deleting *v1.ServiceAccount: provisioning-2010-2442/csi-gce-pd-controller-sa May 23 07:00:00.852: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-provisioning-2010 May 23 07:00:00.950: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-provisioning-2010 May 23 07:00:01.052: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-provisioning-2010 May 23 07:00:01.154: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-provisioning-2010 May 23 07:00:01.250: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-provisioning-2010 May 23 07:00:01.353: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-provisioning-2010 May 23 07:00:01.449: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-provisioning-2010 May 23 07:00:01.551: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-provisioning-2010 May 23 07:00:01.650: INFO: deleting *v1.Role: provisioning-2010-2442/csi-gce-pd-leaderelection-role-provisioning-2010 May 23 07:00:01.751: INFO: deleting *v1.RoleBinding: provisioning-2010-2442/csi-gce-pd-controller-leaderelection-binding May 23 07:00:01.850: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-provisioning-2010 May 23 07:00:01.950: INFO: deleting *v1.DaemonSet: provisioning-2010-2442/csi-gce-pd-node May 23 07:00:02.054: INFO: deleting *v1.StatefulSet: provisioning-2010-2442/csi-gce-pd-controller �[1mSTEP�[0m: deleting the driver namespace: provisioning-2010-2442 �[1mSTEP�[0m: Waiting for namespaces [provisioning-2010-2442] to vanish [AfterEach] [Testpattern: Dynamic PV (block volmode)] provisioning test/e2e/framework/framework.go:187 �[1mSTEP�[0m: Collecting events from namespace "provisioning-2010". �[1mSTEP�[0m: Found 0 events. May 23 07:00:08.452: INFO: POD NODE PHASE GRACE CONDITIONS May 23 07:00:08.452: INFO: May 23 07:00:08.493: INFO: Logging node info for node bootstrap-e2e-master May 23 07:00:08.533: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 14639 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:39:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:59:12 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:59:12 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:59:12 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:59:12 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:00:08.534: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 07:00:08.571: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 07:00:08.633: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 07:00:08.633: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 07:00:08.633: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container kube-scheduler ready: true, restart count 0 May 23 07:00:08.633: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 07:00:08.633: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container etcd-container ready: true, restart count 0 May 23 07:00:08.633: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 07:00:08.633: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 07:00:08.633: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:00:08.633: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:00:08.633: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container etcd-container ready: true, restart count 0 May 23 07:00:08.633: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:00:08.633: INFO: Container kube-apiserver ready: true, restart count 0 May 23 07:00:08.851: INFO: Latency metrics for node bootstrap-e2e-master May 23 07:00:08.851: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 07:00:08.891: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 14619 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 06:39:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:39:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:57:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:57:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:57:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:57:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:57:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:57:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:57:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:55:15 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:55:15 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:55:15 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:55:15 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:00:08.892: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 07:00:08.930: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 07:00:09.075: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.075: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:00:09.075: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 07:00:09.075: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:00:09.075: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:00:09.075: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.075: INFO: Container coredns ready: true, restart count 0 May 23 07:00:09.075: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.075: INFO: Container autoscaler ready: true, restart count 0 May 23 07:00:09.075: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.075: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:00:09.276: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 07:00:09.277: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 07:00:09.319: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 14621 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:39:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:56:58 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:56:58 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:56:58 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:56:58 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:56:58 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:56:58 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:56:58 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:58:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:58:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:58:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:58:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:00:09.324: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 07:00:09.362: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 07:00:09.423: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.423: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:00:09.423: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:00:09.423: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:00:09.423: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:00:09.423: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.423: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:00:09.423: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.423: INFO: Container coredns ready: true, restart count 0 May 23 07:00:09.423: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 07:00:09.423: INFO: Container metrics-server ready: true, restart count 0 May 23 07:00:09.423: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 07:00:09.590: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 07:00:09.590: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 07:00:09.630: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 14631 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:45:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 06:39:37 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:56:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:56:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:56:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:56:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:56:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:56:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:56:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:58:50 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:58:50 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:58:50 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:58:50 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:00:09.630: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 07:00:09.668: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 07:00:09.729: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.729: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 07:00:09.729: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.729: INFO: Container default-http-backend ready: true, restart count 0 May 23 07:00:09.729: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.729: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:00:09.729: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:00:09.729: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:00:09.729: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:00:09.729: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:00:09.729: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:00:09.903: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst May 23 07:00:09.903: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "provisioning-2010" for this suite.
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sprovisioning\sshould\sprovision\sstorage\swith\ssnapshot\sdata\ssource\s\[Feature\:VolumeSnapshotDataSource\]$'
test/e2e/storage/testsuites/provisioning.go:208 May 23 07:19:21.295: Unexpected error: <*errors.errorString | 0xc00440b850>: { s: "VolumeSnapshot snapshot-hbnzg is not ready within 5m0s", } VolumeSnapshot snapshot-hbnzg is not ready within 5m0s occurred test/e2e/storage/utils/snapshot.go:86from junit_01.xml
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Creating a kubernetes client May 23 07:13:51.707: INFO: >>> kubeConfig: /workspace/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename provisioning �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] test/e2e/storage/testsuites/provisioning.go:208 �[1mSTEP�[0m: deploying csi gce-pd driver �[1mSTEP�[0m: Building a driver namespace object, basename provisioning-6825 �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace May 23 07:13:52.279: INFO: Found CI service account key at /etc/service-account/service-account.json May 23 07:13:52.279: INFO: Running cp [/etc/service-account/service-account.json /tmp/d411cdae-9de0-4083-aec3-11d421998ecc/cloud-sa.json] May 23 07:13:52.324: INFO: Shredding file /tmp/d411cdae-9de0-4083-aec3-11d421998ecc/cloud-sa.json May 23 07:13:52.324: INFO: Running shred [--remove /tmp/d411cdae-9de0-4083-aec3-11d421998ecc/cloud-sa.json] May 23 07:13:52.361: INFO: File /tmp/d411cdae-9de0-4083-aec3-11d421998ecc/cloud-sa.json successfully shredded May 23 07:13:52.371: INFO: creating *v1.ServiceAccount: provisioning-6825-4701/csi-attacher May 23 07:13:52.412: INFO: creating *v1.ClusterRole: external-attacher-runner-provisioning-6825 May 23 07:13:52.412: INFO: Define cluster role external-attacher-runner-provisioning-6825 May 23 07:13:52.453: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-provisioning-6825 May 23 07:13:52.493: INFO: creating *v1.Role: provisioning-6825-4701/external-attacher-cfg-provisioning-6825 May 23 07:13:52.532: INFO: creating *v1.RoleBinding: provisioning-6825-4701/csi-attacher-role-cfg May 23 07:13:52.649: INFO: creating *v1.ServiceAccount: provisioning-6825-4701/csi-provisioner May 23 07:13:52.696: INFO: creating *v1.ClusterRole: external-provisioner-runner-provisioning-6825 May 23 07:13:52.696: INFO: Define cluster role external-provisioner-runner-provisioning-6825 May 23 07:13:52.737: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-6825 May 23 07:13:52.778: INFO: creating *v1.Role: provisioning-6825-4701/external-provisioner-cfg-provisioning-6825 May 23 07:13:52.817: INFO: creating *v1.RoleBinding: provisioning-6825-4701/csi-provisioner-role-cfg May 23 07:13:52.858: INFO: creating *v1.ServiceAccount: provisioning-6825-4701/csi-gce-pd-controller-sa May 23 07:13:52.897: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-provisioning-6825 May 23 07:13:52.897: INFO: Define cluster role csi-gce-pd-provisioner-role-provisioning-6825 May 23 07:13:52.937: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-provisioning-6825 May 23 07:13:52.977: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-provisioning-6825 May 23 07:13:52.977: INFO: Define cluster role csi-gce-pd-attacher-role-provisioning-6825 May 23 07:13:53.016: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-provisioning-6825 May 23 07:13:53.056: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-provisioning-6825 May 23 07:13:53.056: INFO: Define cluster role csi-gce-pd-resizer-role-provisioning-6825 May 23 07:13:53.095: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-provisioning-6825 May 23 07:13:53.135: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-provisioning-6825 May 23 07:13:53.135: INFO: Define cluster role csi-gce-pd-snapshotter-role-provisioning-6825 May 23 07:13:53.174: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-provisioning-6825 May 23 07:13:53.213: INFO: creating *v1.Role: provisioning-6825-4701/csi-gce-pd-leaderelection-role-provisioning-6825 May 23 07:13:53.254: INFO: creating *v1.RoleBinding: provisioning-6825-4701/csi-gce-pd-controller-leaderelection-binding May 23 07:13:53.294: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-provisioning-6825 May 23 07:13:53.333: INFO: creating *v1.DaemonSet: provisioning-6825-4701/csi-gce-pd-node May 23 07:13:53.377: INFO: creating *v1.StatefulSet: provisioning-6825-4701/csi-gce-pd-controller May 23 07:13:53.697: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-qjt1 May 23 07:13:58.915: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-tls5 May 23 07:13:58.954: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-vtst May 23 07:13:58.992: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 23 07:13:58.992: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil May 23 07:13:58.992: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-68254stft 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:pd.csi.storage.gke.io,Parameters:map[string]string{type: pd-standard,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-6825 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-68254stft,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-6825 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-68254stft,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP�[0m: Creating a StorageClass �[1mSTEP�[0m: [Initialize dataSource]creating a initClaim �[1mSTEP�[0m: starting gcepd-injector May 23 07:13:59.206: INFO: Waiting up to 5m0s for pod "gcepd-injector" in namespace "provisioning-6825" to be "running" May 23 07:13:59.244: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 37.853884ms May 23 07:14:01.284: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2.077320205s May 23 07:14:03.289: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4.08223849s May 23 07:14:05.284: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 6.077261163s May 23 07:14:07.284: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 8.077689229s May 23 07:14:09.283: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 10.076704423s May 23 07:14:11.294: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 12.087483611s May 23 07:14:13.284: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 14.077756573s May 23 07:14:15.284: INFO: Pod "gcepd-injector": Phase="Running", Reason="", readiness=true. Elapsed: 16.077520755s May 23 07:14:15.284: INFO: Pod "gcepd-injector" satisfied condition "running" �[1mSTEP�[0m: Writing text file contents in the container. May 23 07:14:15.284: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.84.103 --kubeconfig=/workspace/.kube/config --namespace=provisioning-6825 exec gcepd-injector --namespace=provisioning-6825 -- /bin/sh -c echo 'Hello from namespace provisioning-6825' > /opt/0/index.html' May 23 07:14:15.827: INFO: stderr: "" May 23 07:14:15.827: INFO: stdout: "" �[1mSTEP�[0m: Checking that text file contents are perfect. May 23 07:14:15.827: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.84.103 --kubeconfig=/workspace/.kube/config --namespace=provisioning-6825 exec gcepd-injector --namespace=provisioning-6825 -- cat /opt/0/index.html' May 23 07:14:16.415: INFO: stderr: "" May 23 07:14:16.415: INFO: stdout: "Hello from namespace provisioning-6825\n" May 23 07:14:16.415: INFO: ExecWithOptions {Command:[/bin/sh -c test -d /opt/0] Namespace:provisioning-6825 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} May 23 07:14:16.415: INFO: >>> kubeConfig: /workspace/.kube/config May 23 07:14:16.416: INFO: ExecWithOptions: Clientset creation May 23 07:14:16.416: INFO: ExecWithOptions: execute(POST https://34.145.84.103/api/v1/namespaces/provisioning-6825/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-d+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true) May 23 07:14:16.726: INFO: ExecWithOptions {Command:[/bin/sh -c test -b /opt/0] Namespace:provisioning-6825 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} May 23 07:14:16.726: INFO: >>> kubeConfig: /workspace/.kube/config May 23 07:14:16.727: INFO: ExecWithOptions: Clientset creation May 23 07:14:16.727: INFO: ExecWithOptions: execute(POST https://34.145.84.103/api/v1/namespaces/provisioning-6825/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-b+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true) �[1mSTEP�[0m: Deleting pod gcepd-injector in namespace provisioning-6825 May 23 07:14:17.086: INFO: Waiting for pod gcepd-injector to disappear May 23 07:14:17.124: INFO: Pod gcepd-injector still exists May 23 07:14:19.125: INFO: Waiting for pod gcepd-injector to disappear May 23 07:14:19.164: INFO: Pod gcepd-injector no longer exists �[1mSTEP�[0m: creating a SnapshotClass �[1mSTEP�[0m: creating a dynamic VolumeSnapshot May 23 07:14:19.286: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-hbnzg to become ready May 23 07:14:19.324: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:21.364: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:23.404: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:25.444: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:27.484: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:29.523: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:31.603: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:33.642: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:35.682: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:37.722: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:39.762: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:41.802: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:43.842: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:45.882: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:47.922: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:49.962: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:52.002: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:54.042: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:56.081: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:14:58.121: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:00.160: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:02.203: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:04.242: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:06.282: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:08.321: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:10.361: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:12.402: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:14.441: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:16.481: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:18.522: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:20.561: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:22.600: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:24.642: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:26.681: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:28.721: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:30.761: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:32.801: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:34.840: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:36.881: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:38.921: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:40.960: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:43.000: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:45.041: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:47.081: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:49.120: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:51.159: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:53.201: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:55.242: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:57.282: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:15:59.322: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:01.361: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:03.401: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:05.441: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:07.480: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:09.519: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:11.559: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:13.598: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:15.638: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:17.678: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:19.719: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:21.758: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:23.799: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:25.840: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:27.879: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:29.919: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:31.959: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:33.999: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:36.039: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:38.099: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:40.139: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:42.181: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:44.220: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:46.261: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:48.301: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:50.341: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:52.404: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:54.444: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:56.484: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:16:58.524: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:00.564: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:02.603: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:04.645: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:06.684: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:08.724: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:10.765: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:12.805: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:14.847: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:16.886: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:18.926: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:20.966: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:23.005: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:25.045: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:27.084: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:29.124: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:31.164: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:33.204: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:35.243: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:37.285: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:39.323: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:41.363: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:43.405: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:45.444: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:47.483: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:49.524: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:51.565: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:53.605: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:55.645: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:57.686: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:17:59.726: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:01.765: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:03.805: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:05.844: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:07.884: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:09.923: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:11.963: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:14.004: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:16.043: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:18.083: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:20.123: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:22.164: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:24.205: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:26.249: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:28.290: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:30.330: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:32.372: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:34.412: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:36.451: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:38.490: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:40.532: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:42.573: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:44.612: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:46.651: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:48.691: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:50.732: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:52.771: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:54.811: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:56.852: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:18:58.893: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:00.934: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:02.974: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:05.015: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:07.056: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:09.095: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:11.136: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:13.175: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:15.214: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:17.254: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:19.294: INFO: VolumeSnapshot snapshot-hbnzg found but is not ready. May 23 07:19:21.294: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 07:19:21.295: FAIL: Unexpected error: <*errors.errorString | 0xc00440b850>: { s: "VolumeSnapshot snapshot-hbnzg is not ready within 5m0s", } VolumeSnapshot snapshot-hbnzg is not ready within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eae40, 0xc00084dda8?}, 0xc00248e290) test/e2e/storage/utils/snapshot.go:86 +0x1ad k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f09442f6580, 0xc00118aea0}, 0xc002e57e00, {{0x717a175, 0x17}, {0x0, 0x0}, {0x7133cd2, 0x9}, {0x0, ...}, ...}, ...) test/e2e/storage/framework/snapshot_resource.go:92 +0x246 k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning(_, {{0xc002264378, 0x11}, {0x712a743, 0x5}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...) test/e2e/storage/testsuites/provisioning.go:1083 +0x41f k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4() test/e2e/storage/testsuites/provisioning.go:227 +0x485 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-6825 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-68254stft,VolumeMode:nil,DataSource:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-hbnzg,},DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} �[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-68254stft 7b5821e7-4681-4dfb-a831-dd8c6bed34c0 17668 0 2022-05-23 07:13:59 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2022-05-23 07:13:59 +0000 UTC FieldsV1 {"f:parameters":{".":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:pd.csi.storage.gke.io,Parameters:map[string]string{type: pd-standard,},ReclaimPolicy:*Delete,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-hkj4l pvc- provisioning-6825 c2f452a0-d307-4bda-bd51-5e38846aa112 18603 0 2022-05-23 07:19:21 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2022-05-23 07:19:21 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:dataSource":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-68254stft,VolumeMode:*Filesystem,DataSource:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-hbnzg,},DataSourceRef:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-hbnzg,},},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},} May 23 07:19:21.419: INFO: Waiting up to 5m0s for pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb" in namespace "provisioning-6825" to be "running" May 23 07:19:21.458: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 38.161949ms May 23 07:19:23.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.077576474s May 23 07:19:25.500: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.080100315s May 23 07:19:27.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.078480022s May 23 07:19:29.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.077897073s May 23 07:19:31.496: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.076774962s May 23 07:19:33.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 12.077332689s May 23 07:19:35.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 14.07806268s May 23 07:19:37.496: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 16.076974358s May 23 07:19:39.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 18.077183437s May 23 07:19:41.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 20.077647099s May 23 07:19:43.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 22.077792425s May 23 07:19:45.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 24.078955649s May 23 07:19:47.499: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 26.07906841s May 23 07:19:49.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 28.077793777s May 23 07:19:51.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 30.078219554s May 23 07:19:53.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 32.077751242s May 23 07:19:55.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 34.077852929s May 23 07:19:57.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 36.078424043s May 23 07:19:59.547: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 38.127902322s May 23 07:20:01.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 40.077817937s May 23 07:20:03.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 42.078783955s May 23 07:20:05.499: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 44.079202792s May 23 07:20:07.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 46.078100223s May 23 07:20:09.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 48.078035684s May 23 07:20:11.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 50.077072119s May 23 07:20:13.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 52.078491798s May 23 07:20:15.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 54.078856512s May 23 07:20:17.499: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 56.079145273s May 23 07:20:19.498: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 58.078170328s May 23 07:20:21.496: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.077064576s May 23 07:20:23.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.077465174s May 23 07:20:25.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.077648071s May 23 07:20:27.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.077265428s May 23 07:20:29.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.077251999s May 23 07:20:31.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.077209704s May 23 07:20:33.499: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.079861118s May 23 07:20:35.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.096669063s May 23 07:20:37.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.09655261s May 23 07:20:39.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.097683085s May 23 07:20:41.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.096680383s May 23 07:20:43.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.097460088s May 23 07:20:45.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.096669987s May 23 07:20:47.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.097161258s May 23 07:20:49.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.097733935s May 23 07:20:51.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.09653144s May 23 07:20:53.553: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.13347948s May 23 07:20:55.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.096548321s May 23 07:20:57.497: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.077816701s May 23 07:20:59.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.096934891s May 23 07:21:01.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.097403278s May 23 07:21:03.536: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.116184422s May 23 07:21:05.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.097487884s May 23 07:21:07.520: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.100560174s May 23 07:21:09.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.097303502s May 23 07:21:11.496: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.076621158s May 23 07:21:13.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.097598942s May 23 07:21:15.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.096893107s May 23 07:21:17.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.097668778s May 23 07:21:19.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.09716213s May 23 07:21:21.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.096892026s May 23 07:21:23.518: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.098757159s May 23 07:21:25.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.096503611s May 23 07:21:27.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.096995427s May 23 07:21:29.515: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.096029217s May 23 07:21:31.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.097037488s May 23 07:21:33.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.096819077s May 23 07:21:35.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.097627259s May 23 07:21:37.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.0972947s May 23 07:21:39.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.096676704s May 23 07:21:41.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.096262408s May 23 07:21:43.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.09667423s May 23 07:21:45.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.096381348s May 23 07:21:47.518: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.09872783s May 23 07:21:49.526: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.106163923s May 23 07:21:51.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.096675834s May 23 07:21:53.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.097314591s May 23 07:21:55.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.09699615s May 23 07:21:57.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.098050521s May 23 07:21:59.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.096515288s May 23 07:22:01.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.096752551s May 23 07:22:03.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.096791342s May 23 07:22:05.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.09668129s May 23 07:22:07.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.09718047s May 23 07:22:09.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.097607729s May 23 07:22:11.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.097303893s May 23 07:22:13.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.096454133s May 23 07:22:15.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.097348572s May 23 07:22:17.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.097201432s May 23 07:22:19.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.097169534s May 23 07:22:21.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.096885734s May 23 07:22:23.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.096898423s May 23 07:22:25.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.097041544s May 23 07:22:27.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.096894334s May 23 07:22:29.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.097480849s May 23 07:22:31.519: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.099892723s May 23 07:22:33.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.097279799s May 23 07:22:35.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.097092756s May 23 07:22:37.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.096573127s May 23 07:22:39.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.097139548s May 23 07:22:41.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.09786316s May 23 07:22:43.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.096659816s May 23 07:22:45.530: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.110542926s May 23 07:22:47.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.096350535s May 23 07:22:49.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.097291263s May 23 07:22:51.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.096755521s May 23 07:22:53.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.097506271s May 23 07:22:55.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.097099761s May 23 07:22:57.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.096564757s May 23 07:22:59.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.09670452s May 23 07:23:01.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.097088823s May 23 07:23:03.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.096899048s May 23 07:23:05.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.097523611s May 23 07:23:07.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.097037509s May 23 07:23:09.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.09690125s May 23 07:23:11.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.096094662s May 23 07:23:13.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.096837309s May 23 07:23:15.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.09668798s May 23 07:23:17.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.097324152s May 23 07:23:19.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.09684316s May 23 07:23:21.519: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.099095593s May 23 07:23:23.519: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.099536037s May 23 07:23:25.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.097709841s May 23 07:23:27.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.097112235s May 23 07:23:29.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.097620638s May 23 07:23:31.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.097178055s May 23 07:23:33.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.097311861s May 23 07:23:35.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.096915803s May 23 07:23:37.518: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.09864239s May 23 07:23:39.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.097079842s May 23 07:23:41.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.096492134s May 23 07:23:43.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.097211978s May 23 07:23:45.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.096502893s May 23 07:23:47.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.097516758s May 23 07:23:49.520: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.100070698s May 23 07:23:51.520: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.100396195s May 23 07:23:53.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.096716253s May 23 07:23:55.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.096726484s May 23 07:23:57.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.097255053s May 23 07:23:59.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.096653396s May 23 07:24:01.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.096938782s May 23 07:24:03.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.097201667s May 23 07:24:05.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.097460497s May 23 07:24:07.525: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.10598057s May 23 07:24:09.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.097800979s May 23 07:24:11.527: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.107263538s May 23 07:24:13.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.096839232s May 23 07:24:15.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.097009225s May 23 07:24:17.518: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.098253697s May 23 07:24:19.516: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.096483307s May 23 07:24:21.517: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.097547345s May 23 07:24:21.617: INFO: Pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.197232628s May 23 07:24:21.618: INFO: Timed out while waiting for pod provisioning-6825/pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb to be running. Last observed as: <*v1.Pod | 0xc00395c800>: { TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb", GenerateName: "", Namespace: "provisioning-6825", SelfLink: "", UID: "075e8140-6be6-425e-9c0b-1149ef61cbf4", ResourceVersion: "18604", Generation: 0, CreationTimestamp: { Time: 2022-05-23T07:19:21Z, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: nil, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: 2022-05-23T07:19:21Z, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/mnt/volume1\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, ], }, Spec: { Volumes: [ { Name: "volume1", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "pvc-hkj4l", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-vr22p", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: nil, DownwardAPI: nil, ConfigMap: nil, ServiceAccountToken: { Audience: "", ExpirationSeconds: 3607, Path: "token", }, }, { Secret: nil, DownwardAPI: nil, ConfigMap: { LocalObjectReference: {Name: ...}, Items: [...], Optional: nil, }, ServiceAccountToken: nil, }, { Secret: nil, DownwardAPI: {Items: [...]}, ConfigMap: nil, ServiceAccountToken: nil, }, ], DefaultMode: 420, }, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, ], InitContainers: nil, Containers: [ { Name: "write-pod", Image: "k8s.gcr.io/e2e-test-images/busybox:1.29-2", Command: [ "/bin/sh", "-c", "trap exit TERM; while true; do sleep 1; done", ], Args: nil, WorkingDir: "", Ports: nil, EnvFrom: nil, Env: nil, Resources: {Limits: nil, Requests: nil}, VolumeMounts: [ { Name: "volume1", ReadOnly: false, MountPath: "/mnt/volume1", SubPath: "", MountPropagation: nil, SubPathExpr: "", }, { Name: "kube-api-access-vr22p", ReadOnly: true, MountPath: "/var/run/secrets/kubernetes.io/serviceaccount", SubPath: "", MountPropagation: nil, SubPathExpr: "", }, ], VolumeDevices: nil, LivenessProbe: nil, ReadinessProbe: nil, StartupProbe: nil, Lifecycle: nil, TerminationMessagePath: "/dev/termination-log", TerminationMessagePolicy: "File", ImagePullPolicy: "IfNotPresent", SecurityContext: { Capabilities: nil, Privileged: false, SELinuxOptions: nil, WindowsOptions: nil, RunAsUser: nil, RunAsGroup: nil, RunAsNonRoot: nil, ReadOnlyRootFilesystem: nil, AllowPrivilegeEscalation: nil, ProcMount: nil, SeccompProfile: nil, }, Stdin: false, StdinOnce: false, TTY: false, }, ], EphemeralContainers: nil, RestartPolicy: "OnFailure", TerminationGracePeriodSeconds: 30, ActiveDeadlineSeconds: nil, DNSPolicy: "ClusterFirst", NodeSelector: nil, ServiceAccountName: "default", DeprecatedServiceAccount: "default", AutomountServiceAccountToken: nil, NodeName: "", HostNetwork: false, HostPID: false, HostIPC: false, ShareProcessNamespace: nil, SecurityContext: { SELinuxOptions: nil, WindowsOptions: nil, RunAsUser: nil, RunAsGroup: nil, RunAsNonRoot: nil, SupplementalGroups: nil, FSGroup: 1000, Sysctls: nil, FSGroupChangePolicy: nil, SeccompProfile: nil, }, ImagePullSecrets: nil, Hostname: "", Subdomain: "", Affinity: nil, SchedulerName: "default-scheduler", Tolerations: [ { Key: "node.kubernetes.io/not-ready", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, { Key: "node.kubernetes.io/unreachable", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, ], HostAliases: nil, PriorityClassName: "", Priority: 0, DNSConfig: nil, ReadinessGates: nil, RuntimeClassName: nil, EnableServiceLinks: true, PreemptionPolicy: "PreemptLowerPriority", Overhead: nil, TopologySpreadConstraints: nil, SetHostnameAsFQDN: nil, OS: nil, }, Status: { Phase: "Pending", Conditions: nil, Message: "", Reason: "", NominatedNodeName: "", HostIP: "", PodIP: "", PodIPs: nil, StartTime: nil, InitContainerStatuses: nil, ContainerStatuses: nil, QOSClass: "BestEffort", EphemeralContainerStatuses: nil, }, } May 23 07:24:21.619: FAIL: Unexpected error: <*errors.errorString | 0xc001f7f100>: { s: "pod \"pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb\" is not Running: timed out while waiting for pod provisioning-6825/pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb to be running", } pod "pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb" is not Running: timed out while waiting for pod provisioning-6825/pod-d5cf9aa3-3407-4c20-8dd1-b285fbbaadeb to be running occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning({{0x7a45528, 0xc002504180}, 0xc000ebc380, 0xc003845180, 0xc003845500, 0xc0045d5e00, {0x0, 0x0}, {0x0, 0x0, ...}, ...}) test/e2e/storage/testsuites/provisioning.go:624 +0x808 k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4() test/e2e/storage/testsuites/provisioning.go:243 +0x618 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f May 23 07:24:21.620: INFO: deleting claim "provisioning-6825"/"pvc-hkj4l" May 23 07:24:21.719: INFO: deleting initClaim "provisioning-6825"/"pvc-km9ch" May 23 07:24:21.819: INFO: deleting snapshot "provisioning-6825"/"snapshot-hbnzg" May 23 07:24:21.858: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 readyToUse:false] May 23 07:24:21.858: INFO: snapshotContentName snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 May 23 07:24:21.938: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 to be deleted May 23 07:24:21.987: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:24.032: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:26.072: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:28.112: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:30.152: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:32.192: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:34.235: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:36.275: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:38.315: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:40.355: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:42.397: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:44.438: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:46.479: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:48.519: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:50.558: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:52.598: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:54.639: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:56.683: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:24:58.723: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:00.762: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:02.802: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:04.888: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:06.927: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:08.970: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:11.012: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:13.056: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:15.095: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:17.136: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:19.191: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:21.231: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:23.270: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:25.310: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:27.354: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:29.394: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:31.433: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:33.473: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:35.712: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:37.752: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:39.793: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:41.832: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:43.872: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:45.912: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:47.953: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:49.991: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:52.039: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:54.078: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:56.121: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:25:58.160: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:00.202: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:02.243: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:04.283: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:06.369: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:08.408: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:10.448: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:12.489: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:14.528: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:16.572: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:18.612: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:20.653: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:22.694: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:24.735: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:26.774: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:28.814: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:30.854: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:32.894: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:34.935: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:36.975: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:39.018: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:41.059: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:43.102: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:45.141: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:47.184: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:49.224: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:51.297: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:53.337: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:55.375: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:57.418: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:26:59.458: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:01.498: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:03.538: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:05.593: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:07.632: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:09.672: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:11.711: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:13.751: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:15.791: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:17.832: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:19.870: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:21.911: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:23.951: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:25.999: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:28.040: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:30.081: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:32.122: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:34.163: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:36.204: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:38.246: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:40.308: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:42.380: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:44.420: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:46.460: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:48.503: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:50.544: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:52.586: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:54.625: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:56.667: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:27:58.707: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:00.748: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:02.789: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:04.830: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:06.871: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:08.911: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:10.950: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:12.990: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:15.035: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:17.076: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:19.118: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:21.163: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:23.205: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:25.246: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:27.287: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:29.328: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:31.373: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:33.412: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:35.452: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:37.492: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:39.576: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:41.625: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:43.673: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:45.729: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:47.772: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:49.816: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:51.857: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:53.904: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:55.944: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:28:57.992: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:00.040: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:02.080: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:04.125: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:06.168: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:08.212: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:10.259: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:12.302: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:14.347: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:16.438: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:18.483: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:20.527: INFO: volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 has been found and is not deleted May 23 07:29:22.528: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 07:29:22.530: FAIL: Unexpected error: <*errors.errorString | 0xc003d2f3c0>: { s: "volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 is not deleted within 5m0s", } volumesnapshotcontents snapcontent-b2d437ec-2e6e-48e5-83ab-32d3441dcc90 is not deleted within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/framework.(*SnapshotResource).CleanupResource(0xc0004bb340, 0xc000ebc380) test/e2e/storage/framework/snapshot_resource.go:204 +0xefd k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning.func1() test/e2e/storage/testsuites/provisioning.go:1098 +0x294 panic({0x6c82880, 0xc003300b40}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x6309f00, 0x79de820}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc00232ef00, 0x1c5}, {0xc003eb8888?, 0x7126f4b?, 0xc003eb88a8?}) test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc00423a380, 0x1b0}, {0xc0044f6f80?, 0xc00423a380?, 0xc002f7e1a0?}) test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc003eb89f0, {0x7a02ab8, 0xa670820}, 0x0, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc003eb89f0, {0x7a02ab8, 0xa670820}, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x7a45528?, {0x79e2440?, 0xc001f7f100?}, {0x0?, 0x2?, 0x0?}) test/e2e/framework/expect.go:46 +0x9d k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...) test/e2e/framework/expect.go:40 k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning({{0x7a45528, 0xc002504180}, 0xc000ebc380, 0xc003845180, 0xc003845500, 0xc0045d5e00, {0x0, 0x0}, {0x0, 0x0, ...}, ...}) test/e2e/storage/testsuites/provisioning.go:624 +0x808 k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4() test/e2e/storage/testsuites/provisioning.go:243 +0x618 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: deleting the test namespace: provisioning-6825 �[1mSTEP�[0m: Waiting for namespaces [provisioning-6825] to vanish May 23 07:34:22.767: INFO: error deleting namespace provisioning-6825: timed out waiting for the condition �[1mSTEP�[0m: uninstalling csi gce-pd driver May 23 07:34:22.768: INFO: deleting *v1.ServiceAccount: provisioning-6825-4701/csi-attacher May 23 07:34:22.869: INFO: deleting *v1.ClusterRole: external-attacher-runner-provisioning-6825 May 23 07:34:22.968: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-provisioning-6825 May 23 07:34:23.068: INFO: deleting *v1.Role: provisioning-6825-4701/external-attacher-cfg-provisioning-6825 May 23 07:34:23.167: INFO: deleting *v1.RoleBinding: provisioning-6825-4701/csi-attacher-role-cfg May 23 07:34:23.268: INFO: deleting *v1.ServiceAccount: provisioning-6825-4701/csi-provisioner May 23 07:34:23.367: INFO: deleting *v1.ClusterRole: external-provisioner-runner-provisioning-6825 May 23 07:34:23.470: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-6825 May 23 07:34:23.568: INFO: deleting *v1.Role: provisioning-6825-4701/external-provisioner-cfg-provisioning-6825 May 23 07:34:23.670: INFO: deleting *v1.RoleBinding: provisioning-6825-4701/csi-provisioner-role-cfg May 23 07:34:23.767: INFO: deleting *v1.ServiceAccount: provisioning-6825-4701/csi-gce-pd-controller-sa May 23 07:34:23.867: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-provisioning-6825 May 23 07:34:23.969: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-provisioning-6825 May 23 07:34:24.068: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-provisioning-6825 May 23 07:34:24.168: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-provisioning-6825 May 23 07:34:24.268: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-provisioning-6825 May 23 07:34:24.368: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-provisioning-6825 May 23 07:34:24.469: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-provisioning-6825 May 23 07:34:24.569: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-provisioning-6825 May 23 07:34:24.668: INFO: deleting *v1.Role: provisioning-6825-4701/csi-gce-pd-leaderelection-role-provisioning-6825 May 23 07:34:24.771: INFO: deleting *v1.RoleBinding: provisioning-6825-4701/csi-gce-pd-controller-leaderelection-binding May 23 07:34:24.869: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-provisioning-6825 May 23 07:34:24.968: INFO: deleting *v1.DaemonSet: provisioning-6825-4701/csi-gce-pd-node May 23 07:34:25.069: INFO: deleting *v1.StatefulSet: provisioning-6825-4701/csi-gce-pd-controller �[1mSTEP�[0m: deleting the driver namespace: provisioning-6825-4701 �[1mSTEP�[0m: Waiting for namespaces [provisioning-6825-4701] to vanish [AfterEach] [Testpattern: Dynamic PV (default fs)] provisioning test/e2e/framework/framework.go:187 �[1mSTEP�[0m: Collecting events from namespace "provisioning-6825". �[1mSTEP�[0m: Found 0 events. May 23 07:34:31.412: INFO: POD NODE PHASE GRACE CONDITIONS May 23 07:34:31.413: INFO: May 23 07:34:31.453: INFO: Logging node info for node bootstrap-e2e-master May 23 07:34:31.493: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 21194 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:13:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:29:47 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:29:47 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:29:47 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:29:47 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:34:31.493: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 07:34:31.531: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 07:34:31.587: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.587: INFO: Container etcd-container ready: true, restart count 0 May 23 07:34:31.587: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.587: INFO: Container kube-apiserver ready: true, restart count 0 May 23 07:34:31.587: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.587: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 07:34:31.587: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.587: INFO: Container etcd-container ready: true, restart count 0 May 23 07:34:31.587: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.587: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 07:34:31.587: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 07:34:31.587: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:34:31.587: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:34:31.587: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.587: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 07:34:31.587: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.588: INFO: Container kube-scheduler ready: true, restart count 0 May 23 07:34:31.588: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.588: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 07:34:31.761: INFO: Latency metrics for node bootstrap-e2e-master May 23 07:34:31.762: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 07:34:31.802: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 21205 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 07:14:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:14:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:32:04 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:32:04 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:32:04 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:32:04 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:32:04 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:32:04 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:32:04 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:29:45 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:29:45 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:29:45 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:29:45 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:f5241226198f5a54d22540acf2b3933ea0f49458f90c51fc75833d0c428687b8 k8s.gcr.io/e2e-test-images/agnhost:2.36],SizeBytes:50655495,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:34:31.802: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 07:34:31.841: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 07:34:31.909: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.909: INFO: Container coredns ready: true, restart count 0 May 23 07:34:31.909: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.909: INFO: Container autoscaler ready: true, restart count 0 May 23 07:34:31.909: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.909: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:34:31.909: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 07:34:31.909: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:34:31.909: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 07:34:31.909: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:34:31.909: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:34:32.090: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 07:34:32.090: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 07:34:32.131: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 21204 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 07:13:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:13:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:32:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:32:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:32:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:32:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:32:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:32:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:32:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:33:51 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:33:51 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:33:51 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:33:51 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:34:32.131: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 07:34:32.170: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 07:34:32.247: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:34:32.247: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:34:32.247: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:34:32.247: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:34:32.247: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:34:32.247: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 07:34:32.247: INFO: Container coredns ready: true, restart count 0 May 23 07:34:32.247: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 07:34:32.247: INFO: Container metrics-server ready: true, restart count 0 May 23 07:34:32.247: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 07:34:32.247: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:34:32.247: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:34:32.429: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 07:34:32.430: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 07:34:32.472: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 21192 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 07:00:18 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:13:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:31:59 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:31:59 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:31:59 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:31:59 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:31:59 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:31:59 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:31:59 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:31:40 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:31:40 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:31:40 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:31:40 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:34:32.472: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 07:34:32.510: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 07:34:32.569: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:34:32.569: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 07:34:32.569: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:34:32.569: INFO: Container default-http-backend ready: true, restart count 0 May 23 07:34:32.569: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:34:32.569: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:34:32.569: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:34:32.569: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:34:32.569: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:34:32.569: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:34:32.569: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:34:32.733: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst May 23 07:34:32.733: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "provisioning-6825" for this suite.
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sDynamic\sSnapshot\s\(delete\spolicy\)\]\ssnapshottable\[Feature\:VolumeSnapshotDataSource\]\svolume\ssnapshot\scontroller\s\sshould\scheck\ssnapshot\sfields\,\scheck\srestore\scorrectly\sworks\safter\smodifying\ssource\sdata\,\scheck\sdeletion\s\(persistent\)$'
test/e2e/storage/testsuites/snapshottable.go:278 May 23 05:52:47.276: Unexpected error: <*errors.errorString | 0xc000da2180>: { s: "VolumeSnapshot snapshot-nzh7f is not ready within 5m0s", } VolumeSnapshot snapshot-nzh7f is not ready within 5m0s occurred test/e2e/storage/utils/snapshot.go:86from junit_01.xml
[BeforeEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Creating a kubernetes client May 23 05:47:02.626: INFO: >>> kubeConfig: /workspace/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename snapshotting �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent) test/e2e/storage/testsuites/snapshottable.go:278 �[1mSTEP�[0m: deploying csi gce-pd driver �[1mSTEP�[0m: Building a driver namespace object, basename snapshotting-4352 �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace May 23 05:47:03.250: INFO: Found CI service account key at /etc/service-account/service-account.json May 23 05:47:03.250: INFO: Running cp [/etc/service-account/service-account.json /tmp/c9cccd65-fd95-465e-b222-94896410a6b7/cloud-sa.json] May 23 05:47:03.293: INFO: Shredding file /tmp/c9cccd65-fd95-465e-b222-94896410a6b7/cloud-sa.json May 23 05:47:03.293: INFO: Running shred [--remove /tmp/c9cccd65-fd95-465e-b222-94896410a6b7/cloud-sa.json] May 23 05:47:03.331: INFO: File /tmp/c9cccd65-fd95-465e-b222-94896410a6b7/cloud-sa.json successfully shredded May 23 05:47:03.337: INFO: creating *v1.ServiceAccount: snapshotting-4352-2783/csi-attacher May 23 05:47:03.378: INFO: creating *v1.ClusterRole: external-attacher-runner-snapshotting-4352 May 23 05:47:03.378: INFO: Define cluster role external-attacher-runner-snapshotting-4352 May 23 05:47:03.417: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-4352 May 23 05:47:03.456: INFO: creating *v1.Role: snapshotting-4352-2783/external-attacher-cfg-snapshotting-4352 May 23 05:47:03.496: INFO: creating *v1.RoleBinding: snapshotting-4352-2783/csi-attacher-role-cfg May 23 05:47:03.536: INFO: creating *v1.ServiceAccount: snapshotting-4352-2783/csi-provisioner May 23 05:47:03.577: INFO: creating *v1.ClusterRole: external-provisioner-runner-snapshotting-4352 May 23 05:47:03.577: INFO: Define cluster role external-provisioner-runner-snapshotting-4352 May 23 05:47:03.617: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-4352 May 23 05:47:03.656: INFO: creating *v1.Role: snapshotting-4352-2783/external-provisioner-cfg-snapshotting-4352 May 23 05:47:03.695: INFO: creating *v1.RoleBinding: snapshotting-4352-2783/csi-provisioner-role-cfg May 23 05:47:03.735: INFO: creating *v1.ServiceAccount: snapshotting-4352-2783/csi-gce-pd-controller-sa May 23 05:47:03.775: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-4352 May 23 05:47:03.775: INFO: Define cluster role csi-gce-pd-provisioner-role-snapshotting-4352 May 23 05:47:03.815: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-4352 May 23 05:47:03.856: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-4352 May 23 05:47:03.856: INFO: Define cluster role csi-gce-pd-attacher-role-snapshotting-4352 May 23 05:47:03.895: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-4352 May 23 05:47:03.935: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-4352 May 23 05:47:03.935: INFO: Define cluster role csi-gce-pd-resizer-role-snapshotting-4352 May 23 05:47:03.975: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-4352 May 23 05:47:04.014: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-4352 May 23 05:47:04.014: INFO: Define cluster role csi-gce-pd-snapshotter-role-snapshotting-4352 May 23 05:47:04.055: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-4352 May 23 05:47:04.094: INFO: creating *v1.Role: snapshotting-4352-2783/csi-gce-pd-leaderelection-role-snapshotting-4352 May 23 05:47:04.133: INFO: creating *v1.RoleBinding: snapshotting-4352-2783/csi-gce-pd-controller-leaderelection-binding May 23 05:47:04.173: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-4352 May 23 05:47:04.212: INFO: creating *v1.DaemonSet: snapshotting-4352-2783/csi-gce-pd-node May 23 05:47:04.260: INFO: creating *v1.StatefulSet: snapshotting-4352-2783/csi-gce-pd-controller May 23 05:47:04.445: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-qjt1 May 23 05:47:14.239: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-tls5 May 23 05:47:14.277: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-vtst May 23 05:47:16.353: INFO: Creating resource for dynamic PV May 23 05:47:16.353: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(pd.csi.storage.gke.io) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass snapshotting-435248nqz �[1mSTEP�[0m: creating a claim May 23 05:47:16.393: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: [init] starting a pod to use the claim �[1mSTEP�[0m: [init] check pod success May 23 05:47:16.570: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-65j79" in namespace "snapshotting-4352" to be "Succeeded or Failed" May 23 05:47:16.608: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 38.636899ms May 23 05:47:18.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 2.077982719s May 23 05:47:20.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 4.077914961s May 23 05:47:22.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 6.077776705s May 23 05:47:24.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 8.078304193s May 23 05:47:26.647: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 10.077557636s May 23 05:47:28.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 12.078391699s May 23 05:47:30.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 14.077977075s May 23 05:47:32.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 16.07834829s May 23 05:47:34.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Pending", Reason="", readiness=false. Elapsed: 18.078070614s May 23 05:47:36.648: INFO: Pod "pvc-snapshottable-tester-65j79": Phase="Succeeded", Reason="", readiness=false. Elapsed: 20.078671267s �[1mSTEP�[0m: Saw pod success May 23 05:47:36.649: INFO: Pod "pvc-snapshottable-tester-65j79" satisfied condition "Succeeded or Failed" �[1mSTEP�[0m: [init] checking the claim May 23 05:47:36.687: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pd.csi.storage.gke.iodbrmg] to have phase Bound May 23 05:47:36.726: INFO: PersistentVolumeClaim pd.csi.storage.gke.iodbrmg found and phase=Bound (38.418274ms) �[1mSTEP�[0m: [init] checking the PV �[1mSTEP�[0m: [init] deleting the pod May 23 05:47:36.856: INFO: Pod pvc-snapshottable-tester-65j79 has the following logs: May 23 05:47:36.856: INFO: Deleting pod "pvc-snapshottable-tester-65j79" in namespace "snapshotting-4352" May 23 05:47:36.902: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-65j79" to be fully deleted �[1mSTEP�[0m: [init] waiting until the node=bootstrap-e2e-minion-group-qjt1 is not using the volume=projects/kubernetes-gci-es-logging/zones/us-west1-b/disks/pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d May 23 05:47:36.980: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/kubernetes-gci-es-logging/zones/us-west1-b/disks/pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d] May 23 05:47:39.020: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/kubernetes-gci-es-logging/zones/us-west1-b/disks/pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d] May 23 05:47:41.061: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/kubernetes-gci-es-logging/zones/us-west1-b/disks/pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d] May 23 05:47:43.102: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/kubernetes-gci-es-logging/zones/us-west1-b/disks/pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d] May 23 05:47:45.142: INFO: current volumes in use: [] May 23 05:47:45.142: INFO: WaitUntil finished successfully after 8.202000109s �[1mSTEP�[0m: creating a SnapshotClass �[1mSTEP�[0m: creating a dynamic VolumeSnapshot May 23 05:47:45.264: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-nzh7f to become ready May 23 05:47:45.308: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:47:47.348: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:47:49.387: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:47:51.427: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:47:53.466: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:47:55.505: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:47:57.545: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:47:59.585: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:01.624: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:03.663: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:05.703: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:07.743: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:09.783: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:11.823: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:13.863: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:15.902: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:17.943: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:19.982: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:22.051: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:24.091: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:26.131: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:28.171: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:30.210: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:32.249: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:34.289: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:36.328: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:38.367: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:40.408: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:42.447: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:44.486: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:46.525: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:48.565: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:50.604: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:52.644: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:54.685: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:56.725: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:48:58.765: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:00.806: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:02.847: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:04.887: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:06.927: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:08.967: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:11.007: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:13.047: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:15.086: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:17.126: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:19.165: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:21.218: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:23.258: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:25.298: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:27.340: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:29.380: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:31.420: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:33.459: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:35.500: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:37.543: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:39.583: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:41.623: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:43.662: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:45.702: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:47.742: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:49.782: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:51.821: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:53.860: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:55.901: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:57.941: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:49:59.981: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:02.041: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:04.080: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:06.120: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:08.161: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:10.202: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:12.244: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:14.284: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:16.323: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:18.362: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:20.402: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:22.443: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:24.483: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:26.522: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:28.561: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:30.602: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:32.641: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:34.681: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:36.720: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:38.759: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:40.800: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:42.840: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:44.879: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:46.918: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:48.957: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:50.996: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:53.037: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:55.077: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:57.118: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:50:59.157: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:01.196: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:03.236: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:05.275: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:07.317: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:09.358: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:11.398: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:13.438: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:15.477: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:17.517: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:19.557: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:21.596: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:23.641: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:25.681: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:27.720: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:29.760: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:31.800: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:33.841: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:35.883: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:37.925: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:39.982: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:42.022: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:44.062: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:46.101: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:48.140: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:50.179: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:52.219: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:54.258: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:56.297: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:51:58.336: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:00.377: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:02.420: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:04.460: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:06.502: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:08.541: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:10.586: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:12.626: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:14.666: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:16.706: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:18.745: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:20.785: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:22.826: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:24.870: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:26.915: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:28.955: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:30.994: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:33.036: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:35.075: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:37.116: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:39.156: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:41.195: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:43.236: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:45.275: INFO: VolumeSnapshot snapshot-nzh7f found but is not ready. May 23 05:52:47.276: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 05:52:47.276: FAIL: Unexpected error: <*errors.errorString | 0xc000da2180>: { s: "VolumeSnapshot snapshot-nzh7f is not ready within 5m0s", } VolumeSnapshot snapshot-nzh7f is not ready within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eae40, 0xc004044790?}, 0xc004045700) test/e2e/storage/utils/snapshot.go:86 +0x1ad k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f09442f6580, 0xc00118aea0}, 0xc0026a0f00, {{0x71bab16, 0x20}, {0x0, 0x0}, {0x7133cd2, 0x9}, {0x0, ...}, ...}, ...) test/e2e/storage/framework/snapshot_resource.go:92 +0x246 k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2() test/e2e/storage/testsuites/snapshottable.go:359 +0xb3d k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: checking the snapshot �[1mSTEP�[0m: checking the SnapshotContent �[1mSTEP�[0m: Modifying source data test �[1mSTEP�[0m: modifying the data in the source PVC May 23 05:52:47.406: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-lllc9" in namespace "snapshotting-4352" to be "Succeeded or Failed" May 23 05:52:47.453: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Pending", Reason="", readiness=false. Elapsed: 46.506716ms May 23 05:52:49.492: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.085943484s May 23 05:52:51.492: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.085739198s May 23 05:52:53.492: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.086149528s May 23 05:52:55.492: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.085539033s May 23 05:52:57.492: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.086123471s May 23 05:52:59.491: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Pending", Reason="", readiness=false. Elapsed: 12.085207744s May 23 05:53:01.491: INFO: Pod "pvc-snapshottable-data-tester-lllc9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 14.085004182s �[1mSTEP�[0m: Saw pod success May 23 05:53:01.491: INFO: Pod "pvc-snapshottable-data-tester-lllc9" satisfied condition "Succeeded or Failed" May 23 05:53:01.587: INFO: Pod pvc-snapshottable-data-tester-lllc9 has the following logs: May 23 05:53:01.587: INFO: Deleting pod "pvc-snapshottable-data-tester-lllc9" in namespace "snapshotting-4352" May 23 05:53:01.634: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-lllc9" to be fully deleted �[1mSTEP�[0m: creating a pvc from the snapshot �[1mSTEP�[0m: starting a pod to use the snapshot May 23 05:53:01.759: INFO: Waiting up to 15m0s for pod "restored-pvc-tester-s29bt" in namespace "snapshotting-4352" to be "running" May 23 05:53:01.800: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 40.913502ms May 23 05:53:03.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.080030738s May 23 05:53:05.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4.082075604s May 23 05:53:07.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6.079922523s May 23 05:53:09.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8.080504364s May 23 05:53:11.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10.08088462s May 23 05:53:13.842: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12.083486259s May 23 05:53:15.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14.081980403s May 23 05:53:17.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 16.080835304s May 23 05:53:19.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 18.080088701s May 23 05:53:21.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 20.08084284s May 23 05:53:23.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 22.079877031s May 23 05:53:25.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 24.082061123s May 23 05:53:27.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 26.081702488s May 23 05:53:29.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 28.080293247s May 23 05:53:31.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 30.080740644s May 23 05:53:33.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 32.080819385s May 23 05:53:35.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 34.082090578s May 23 05:53:37.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 36.082457206s May 23 05:53:39.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 38.079523638s May 23 05:53:41.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 40.080763776s May 23 05:53:43.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 42.080283524s May 23 05:53:45.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 44.080953144s May 23 05:53:47.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 46.080740499s May 23 05:53:49.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 48.080226543s May 23 05:53:51.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 50.08009428s May 23 05:53:53.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 52.081075833s May 23 05:53:55.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 54.081100676s May 23 05:53:57.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 56.080372561s May 23 05:53:59.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 58.080584494s May 23 05:54:01.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.080805345s May 23 05:54:03.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.079826242s May 23 05:54:05.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.08048633s May 23 05:54:07.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.079582189s May 23 05:54:09.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.07953385s May 23 05:54:11.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.081047011s May 23 05:54:13.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.080581153s May 23 05:54:15.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.08154165s May 23 05:54:17.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.081314931s May 23 05:54:19.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.0804301s May 23 05:54:21.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.080772392s May 23 05:54:23.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.079537976s May 23 05:54:25.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.081661738s May 23 05:54:27.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.095497121s May 23 05:54:29.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.080808684s May 23 05:54:31.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.079664281s May 23 05:54:33.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.081616644s May 23 05:54:35.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.082058264s May 23 05:54:37.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.080812399s May 23 05:54:39.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.080260838s May 23 05:54:41.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.080424828s May 23 05:54:43.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.08166169s May 23 05:54:45.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.081355042s May 23 05:54:47.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.079811254s May 23 05:54:49.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.080042703s May 23 05:54:51.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.079965922s May 23 05:54:53.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.082797641s May 23 05:54:55.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.081995237s May 23 05:54:57.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.081678418s May 23 05:54:59.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.080052617s May 23 05:55:01.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.080594635s May 23 05:55:03.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.079592117s May 23 05:55:05.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.080293778s May 23 05:55:07.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.081378284s May 23 05:55:09.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.080264551s May 23 05:55:11.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.080487333s May 23 05:55:13.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.081113155s May 23 05:55:15.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.081816227s May 23 05:55:17.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.081259869s May 23 05:55:19.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.081707106s May 23 05:55:21.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.081071738s May 23 05:55:23.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.081221872s May 23 05:55:25.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.080822815s May 23 05:55:27.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.081136356s May 23 05:55:29.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.080607609s May 23 05:55:31.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.080594268s May 23 05:55:33.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.081153711s May 23 05:55:35.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.080289478s May 23 05:55:37.842: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.083204369s May 23 05:55:39.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.080057724s May 23 05:55:41.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.080658127s May 23 05:55:43.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.080122133s May 23 05:55:45.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.080638949s May 23 05:55:47.841: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.082009104s May 23 05:55:49.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.079388849s May 23 05:55:51.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.079855378s May 23 05:55:53.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.080934635s May 23 05:55:55.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.080510693s May 23 05:55:57.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.080863383s May 23 05:55:59.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.079703178s May 23 05:56:01.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.080708225s May 23 05:56:03.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.081480485s May 23 05:56:05.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.08088986s May 23 05:56:07.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.080707722s May 23 05:56:09.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.079848178s May 23 05:56:11.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.080745534s May 23 05:56:13.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.081308223s May 23 05:56:15.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.080764927s May 23 05:56:17.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.080000795s May 23 05:56:19.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.079777674s May 23 05:56:21.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.080861154s May 23 05:56:23.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.081004295s May 23 05:56:25.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.080353028s May 23 05:56:27.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.081035189s May 23 05:56:29.844: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.085523346s May 23 05:56:31.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.080440041s May 23 05:56:33.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.081649046s May 23 05:56:35.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.079991766s May 23 05:56:37.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.081692365s May 23 05:56:39.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.080200005s May 23 05:56:41.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.080144883s May 23 05:56:43.846: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.087157353s May 23 05:56:45.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.080836432s May 23 05:56:47.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.081480561s May 23 05:56:49.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.080069558s May 23 05:56:51.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.080106919s May 23 05:56:53.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.080607314s May 23 05:56:55.843: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.084383788s May 23 05:56:57.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.079697798s May 23 05:56:59.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.079754953s May 23 05:57:01.838: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.079431462s May 23 05:57:03.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.092801385s May 23 05:57:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.09340438s May 23 05:57:07.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.093408428s May 23 05:57:09.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.093254234s May 23 05:57:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.092929347s May 23 05:57:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.093168491s May 23 05:57:15.840: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.081225327s May 23 05:57:17.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.093477961s May 23 05:57:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.09333452s May 23 05:57:21.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.093129656s May 23 05:57:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.092964642s May 23 05:57:25.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.080677433s May 23 05:57:27.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.093321355s May 23 05:57:29.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.092654722s May 23 05:57:31.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.093424025s May 23 05:57:33.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.093481868s May 23 05:57:35.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.09291319s May 23 05:57:37.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.094110625s May 23 05:57:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.093427495s May 23 05:57:41.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.093201559s May 23 05:57:43.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.093591647s May 23 05:57:45.906: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.147865551s May 23 05:57:47.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.093094178s May 23 05:57:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.093693102s May 23 05:57:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.09334315s May 23 05:57:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.093716061s May 23 05:57:55.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.093195977s May 23 05:57:57.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.093188486s May 23 05:57:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.093520378s May 23 05:58:01.865: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.105928181s May 23 05:58:03.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m2.092805873s May 23 05:58:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m4.093724973s May 23 05:58:07.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m6.093288884s May 23 05:58:09.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m8.09292049s May 23 05:58:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m10.093562377s May 23 05:58:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m12.093679205s May 23 05:58:15.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m14.093192622s May 23 05:58:17.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m16.092857665s May 23 05:58:19.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m18.094218222s May 23 05:58:21.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m20.092871827s May 23 05:58:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m22.093393235s May 23 05:58:25.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m24.09350839s May 23 05:58:27.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m26.094003858s May 23 05:58:29.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m28.094122379s May 23 05:58:31.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m30.093004671s May 23 05:58:33.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m32.09335313s May 23 05:58:35.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m34.093000865s May 23 05:58:37.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m36.094170747s May 23 05:58:39.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m38.09402066s May 23 05:58:41.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m40.093048676s May 23 05:58:43.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m42.093407815s May 23 05:58:45.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m44.093312571s May 23 05:58:47.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m46.093275372s May 23 05:58:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m48.093480035s May 23 05:58:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m50.093253095s May 23 05:58:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m52.093127053s May 23 05:58:55.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m54.093702991s May 23 05:58:57.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m56.093120033s May 23 05:58:59.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m58.092652923s May 23 05:59:01.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m0.093071579s May 23 05:59:03.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m2.093345263s May 23 05:59:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m4.093825654s May 23 05:59:07.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m6.092789095s May 23 05:59:09.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m8.093064567s May 23 05:59:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m10.093455707s May 23 05:59:13.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m12.093934446s May 23 05:59:15.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m14.094189884s May 23 05:59:17.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m16.097649236s May 23 05:59:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m18.093151089s May 23 05:59:21.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m20.093515135s May 23 05:59:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m22.093045902s May 23 05:59:25.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m24.093297611s May 23 05:59:27.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m26.094044811s May 23 05:59:29.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m28.093501247s May 23 05:59:31.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m30.094177532s May 23 05:59:33.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m32.092514702s May 23 05:59:35.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m34.093436648s May 23 05:59:37.869: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m36.110660177s May 23 05:59:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m38.092978736s May 23 05:59:41.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m40.094656968s May 23 05:59:43.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m42.095206652s May 23 05:59:45.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m44.093839613s May 23 05:59:47.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m46.092500371s May 23 05:59:49.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m48.092630864s May 23 05:59:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m50.093105434s May 23 05:59:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m52.092994212s May 23 05:59:55.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m54.093133419s May 23 05:59:57.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m56.092883338s May 23 05:59:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m58.093281316s May 23 06:00:01.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m0.093090639s May 23 06:00:03.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m2.093553715s May 23 06:00:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m4.093313882s May 23 06:00:07.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m6.093418917s May 23 06:00:09.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m8.093442165s May 23 06:00:11.857: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m10.098150052s May 23 06:00:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m12.093095132s May 23 06:00:15.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m14.094186496s May 23 06:00:17.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m16.092656365s May 23 06:00:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m18.093055714s May 23 06:00:21.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m20.093435637s May 23 06:00:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m22.093109215s May 23 06:00:25.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m24.094073273s May 23 06:00:27.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m26.092978865s May 23 06:00:29.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m28.093072497s May 23 06:00:31.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m30.093037597s May 23 06:00:33.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m32.092638161s May 23 06:00:35.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m34.094562234s May 23 06:00:37.855: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m36.096538011s May 23 06:00:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m38.09340718s May 23 06:00:41.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m40.093678235s May 23 06:00:43.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m42.092685086s May 23 06:00:45.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m44.094173978s May 23 06:00:47.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m46.093055108s May 23 06:00:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m48.093325026s May 23 06:00:51.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m50.09504842s May 23 06:00:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m52.093844159s May 23 06:00:55.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m54.094309876s May 23 06:00:57.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m56.093324251s May 23 06:00:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m58.093704291s May 23 06:01:01.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m0.093143776s May 23 06:01:03.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m2.093094101s May 23 06:01:05.857: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m4.098733893s May 23 06:01:07.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m6.093051106s May 23 06:01:09.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m8.093543202s May 23 06:01:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m10.093252969s May 23 06:01:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m12.093115308s May 23 06:01:15.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m14.093583185s May 23 06:01:17.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m16.093412224s May 23 06:01:19.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m18.092539039s May 23 06:01:21.855: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m20.096360013s May 23 06:01:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m22.093799932s May 23 06:01:25.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m24.094593757s May 23 06:01:27.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m26.093601001s May 23 06:01:29.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m28.093737971s May 23 06:01:31.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m30.092612773s May 23 06:01:33.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m32.092911675s May 23 06:01:35.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m34.094803991s May 23 06:01:37.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m36.093389135s May 23 06:01:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m38.093067654s May 23 06:01:41.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m40.093190069s May 23 06:01:43.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m42.09280272s May 23 06:01:45.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m44.093438461s May 23 06:01:47.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m46.093080159s May 23 06:01:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m48.093690199s May 23 06:01:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m50.093377625s May 23 06:01:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m52.093493622s May 23 06:01:55.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m54.094606671s May 23 06:01:57.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m56.094058988s May 23 06:01:59.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m58.092833855s May 23 06:02:01.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m0.09264424s May 23 06:02:03.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m2.093244215s May 23 06:02:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m4.093266047s May 23 06:02:07.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m6.092872355s May 23 06:02:09.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m8.092970236s May 23 06:02:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m10.093258605s May 23 06:02:13.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m12.092872372s May 23 06:02:15.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m14.09349213s May 23 06:02:17.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m16.093341299s May 23 06:02:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m18.093556256s May 23 06:02:21.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m20.092713851s May 23 06:02:23.864: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m22.105661767s May 23 06:02:25.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m24.093646191s May 23 06:02:27.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m26.092900794s May 23 06:02:29.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m28.095371701s May 23 06:02:31.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m30.094191193s May 23 06:02:33.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m32.093213908s May 23 06:02:35.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m34.094190996s May 23 06:02:37.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m36.094085s May 23 06:02:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m38.09342655s May 23 06:02:41.839: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m40.080224159s May 23 06:02:43.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m42.093505211s May 23 06:02:45.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m44.094416206s May 23 06:02:47.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m46.092518498s May 23 06:02:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m48.09355241s May 23 06:02:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m50.093346246s May 23 06:02:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m52.093489794s May 23 06:02:55.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m54.093608728s May 23 06:02:57.932: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m56.173630484s May 23 06:02:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m58.093127246s May 23 06:03:01.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m0.093406731s May 23 06:03:03.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m2.092867761s May 23 06:03:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m4.093433748s May 23 06:03:07.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m6.093598616s May 23 06:03:09.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m8.092870206s May 23 06:03:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m10.093153076s May 23 06:03:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m12.093167674s May 23 06:03:15.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m14.093840358s May 23 06:03:17.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m16.09271753s May 23 06:03:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m18.09338455s May 23 06:03:21.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m20.094023223s May 23 06:03:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m22.093812962s May 23 06:03:25.855: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m24.096864229s May 23 06:03:27.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m26.093972281s May 23 06:03:29.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m28.093895394s May 23 06:03:31.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m30.093613375s May 23 06:03:33.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m32.092694057s May 23 06:03:35.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m34.093079966s May 23 06:03:37.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m36.093940154s May 23 06:03:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m38.093212369s May 23 06:03:41.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m40.093891153s May 23 06:03:43.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m42.093870755s May 23 06:03:45.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m44.094149619s May 23 06:03:47.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m46.094456807s May 23 06:03:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m48.093499068s May 23 06:03:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m50.09369865s May 23 06:03:53.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m52.092546153s May 23 06:03:55.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m54.09336748s May 23 06:03:57.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m56.093626545s May 23 06:03:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m58.092895316s May 23 06:04:01.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m0.093785615s May 23 06:04:03.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m2.093062086s May 23 06:04:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m4.093840609s May 23 06:04:07.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m6.094864045s May 23 06:04:09.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m8.092699357s May 23 06:04:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m10.093587749s May 23 06:04:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m12.093328637s May 23 06:04:15.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m14.09367082s May 23 06:04:17.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m16.093102188s May 23 06:04:19.860: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m18.101875952s May 23 06:04:21.861: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m20.102420696s May 23 06:04:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m22.093006819s May 23 06:04:25.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m24.093200489s May 23 06:04:27.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m26.092917635s May 23 06:04:29.859: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m28.100792291s May 23 06:04:31.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m30.092655409s May 23 06:04:33.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m32.092409422s May 23 06:04:35.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m34.094041518s May 23 06:04:37.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m36.097681914s May 23 06:04:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m38.093371561s May 23 06:04:41.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m40.092886365s May 23 06:04:43.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m42.093611225s May 23 06:04:45.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m44.094598637s May 23 06:04:47.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m46.093303743s May 23 06:04:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m48.093639462s May 23 06:04:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m50.093547846s May 23 06:04:53.866: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m52.107025609s May 23 06:04:55.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m54.094051424s May 23 06:04:57.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m56.09356075s May 23 06:04:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m58.093217707s May 23 06:05:01.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m0.093148398s May 23 06:05:03.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m2.093411234s May 23 06:05:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m4.093531526s May 23 06:05:07.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m6.093915651s May 23 06:05:09.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m8.092886691s May 23 06:05:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m10.09358057s May 23 06:05:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m12.093311334s May 23 06:05:15.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m14.093521996s May 23 06:05:17.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m16.093290436s May 23 06:05:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m18.093729974s May 23 06:05:21.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m20.093421094s May 23 06:05:23.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m22.094084546s May 23 06:05:25.860: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m24.101050701s May 23 06:05:27.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m26.093621009s May 23 06:05:29.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m28.093475904s May 23 06:05:31.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m30.093812923s May 23 06:05:33.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m32.093798788s May 23 06:05:35.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m34.093447725s May 23 06:05:37.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m36.093554989s May 23 06:05:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m38.093880479s May 23 06:05:41.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m40.094353649s May 23 06:05:43.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m42.093986429s May 23 06:05:45.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m44.094019941s May 23 06:05:47.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m46.093104866s May 23 06:05:49.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m48.094415882s May 23 06:05:51.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m50.092974281s May 23 06:05:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m52.093805375s May 23 06:05:55.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m54.094471591s May 23 06:05:57.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m56.093877638s May 23 06:05:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m58.093842913s May 23 06:06:01.875: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m0.116086639s May 23 06:06:03.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m2.092804176s May 23 06:06:05.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m4.093708825s May 23 06:06:07.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m6.09407336s May 23 06:06:09.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m8.092793534s May 23 06:06:11.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m10.093542839s May 23 06:06:13.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m12.093673016s May 23 06:06:15.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m14.093591357s May 23 06:06:17.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m16.09362285s May 23 06:06:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m18.093076474s May 23 06:06:21.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m20.093030984s May 23 06:06:23.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m22.096917153s May 23 06:06:25.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m24.097428252s May 23 06:06:27.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m26.093234461s May 23 06:06:29.855: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m28.09605888s May 23 06:06:31.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m30.097407533s May 23 06:06:33.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m32.095517651s May 23 06:06:35.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m34.097364168s May 23 06:06:37.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m36.097328624s May 23 06:06:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m38.093269137s May 23 06:06:41.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m40.097242459s May 23 06:06:43.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m42.09745494s May 23 06:06:45.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m44.097474772s May 23 06:06:47.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m46.097373567s May 23 06:06:49.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m48.097426807s May 23 06:06:51.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m50.094073672s May 23 06:06:53.860: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m52.101310753s May 23 06:06:55.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m54.095493886s May 23 06:06:57.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m56.097385263s May 23 06:06:59.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m58.097409238s May 23 06:07:01.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m0.095766075s May 23 06:07:03.855: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m2.09655164s May 23 06:07:05.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m4.097402755s May 23 06:07:07.844: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m6.085424722s May 23 06:07:09.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m8.097344334s May 23 06:07:11.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m10.094538936s May 23 06:07:13.860: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m12.101409818s May 23 06:07:15.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m14.093892498s May 23 06:07:17.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m16.092692325s May 23 06:07:19.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m18.092928034s May 23 06:07:21.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m20.093764213s May 23 06:07:23.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m22.093569371s May 23 06:07:25.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m24.094291315s May 23 06:07:27.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m26.095554168s May 23 06:07:29.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m28.09376532s May 23 06:07:31.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m30.097583702s May 23 06:07:33.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m32.094052625s May 23 06:07:35.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m34.093914942s May 23 06:07:37.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m36.095217866s May 23 06:07:39.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m38.093528891s May 23 06:07:41.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m40.093327832s May 23 06:07:43.857: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m42.098779166s May 23 06:07:45.862: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m44.103054631s May 23 06:07:47.856: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m46.097719353s May 23 06:07:49.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m48.093133794s May 23 06:07:51.854: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.095731628s May 23 06:07:53.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.093604801s May 23 06:07:55.853: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.094238958s May 23 06:07:57.851: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.092874856s May 23 06:07:59.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.093587239s May 23 06:08:01.852: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.092887249s May 23 06:08:01.952: INFO: Pod "restored-pvc-tester-s29bt": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.193215679s May 23 06:08:01.953: INFO: Timed out while waiting for pod snapshotting-4352/restored-pvc-tester-s29bt to be running. Last observed as: <*v1.Pod | 0xc00206ac00>: { TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "restored-pvc-tester-s29bt", GenerateName: "restored-pvc-tester-", Namespace: "snapshotting-4352", SelfLink: "", UID: "1af51774-b0b9-4829-ba18-9cb18e83f279", ResourceVersion: "4610", Generation: 0, CreationTimestamp: { Time: 2022-05-23T05:53:01Z, }, DeletionTimestamp: nil, DeletionGracePeriodSeconds: nil, Labels: { "app": "restored-pvc-tester", }, Annotations: nil, OwnerReferences: nil, Finalizers: nil, ManagedFields: [ { Manager: "e2e.test", Operation: "Update", APIVersion: "v1", Time: { Time: 2022-05-23T05:53:01Z, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:metadata\":{\"f:generateName\":{},\"f:labels\":{\".\":{},\"f:app\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"volume-tester\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/mnt/test\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"my-volume\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}", }, Subresource: "", }, { Manager: "kube-scheduler", Operation: "Update", APIVersion: "v1", Time: { Time: 2022-05-23T06:03:01Z, }, FieldsType: "FieldsV1", FieldsV1: { Raw: "{\"f:status\":{\"f:conditions\":{\".\":{},\"k:{\\\"type\\\":\\\"PodScheduled\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}}}}", }, Subresource: "status", }, ], }, Spec: { Volumes: [ { Name: "my-volume", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: { ClaimName: "pvc-sp4rr", ReadOnly: false, }, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: nil, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, { Name: "kube-api-access-xsbj2", VolumeSource: { HostPath: nil, EmptyDir: nil, GCEPersistentDisk: nil, AWSElasticBlockStore: nil, GitRepo: nil, Secret: nil, NFS: nil, ISCSI: nil, Glusterfs: nil, PersistentVolumeClaim: nil, RBD: nil, FlexVolume: nil, Cinder: nil, CephFS: nil, Flocker: nil, DownwardAPI: nil, FC: nil, AzureFile: nil, ConfigMap: nil, VsphereVolume: nil, Quobyte: nil, AzureDisk: nil, PhotonPersistentDisk: nil, Projected: { Sources: [ { Secret: nil, DownwardAPI: nil, ConfigMap: nil, ServiceAccountToken: { Audience: "", ExpirationSeconds: 3607, Path: "token", }, }, { Secret: nil, DownwardAPI: nil, ConfigMap: { LocalObjectReference: {Name: ...}, Items: [...], Optional: nil, }, ServiceAccountToken: nil, }, { Secret: nil, DownwardAPI: {Items: [...]}, ConfigMap: nil, ServiceAccountToken: nil, }, ], DefaultMode: 420, }, PortworxVolume: nil, ScaleIO: nil, StorageOS: nil, CSI: nil, Ephemeral: nil, }, }, ], InitContainers: nil, Containers: [ { Name: "volume-tester", Image: "k8s.gcr.io/e2e-test-images/busybox:1.29-2", Command: ["/bin/sh", "-c", "sleep 300"], Args: nil, WorkingDir: "", Ports: nil, EnvFrom: nil, Env: nil, Resources: {Limits: nil, Requests: nil}, VolumeMounts: [ { Name: "my-volume", ReadOnly: false, MountPath: "/mnt/test", SubPath: "", MountPropagation: nil, SubPathExpr: "", }, { Name: "kube-api-access-xsbj2", ReadOnly: true, MountPath: "/var/run/secrets/kubernetes.io/serviceaccount", SubPath: "", MountPropagation: nil, SubPathExpr: "", }, ], VolumeDevices: nil, LivenessProbe: nil, ReadinessProbe: nil, StartupProbe: nil, Lifecycle: nil, TerminationMessagePath: "/dev/termination-log", TerminationMessagePolicy: "File", ImagePullPolicy: "IfNotPresent", SecurityContext: nil, Stdin: false, StdinOnce: false, TTY: false, }, ], EphemeralContainers: nil, RestartPolicy: "Never", TerminationGracePeriodSeconds: 30, ActiveDeadlineSeconds: nil, DNSPolicy: "ClusterFirst", NodeSelector: nil, ServiceAccountName: "default", DeprecatedServiceAccount: "default", AutomountServiceAccountToken: nil, NodeName: "", HostNetwork: false, HostPID: false, HostIPC: false, ShareProcessNamespace: nil, SecurityContext: { SELinuxOptions: nil, WindowsOptions: nil, RunAsUser: nil, RunAsGroup: nil, RunAsNonRoot: nil, SupplementalGroups: nil, FSGroup: nil, Sysctls: nil, FSGroupChangePolicy: nil, SeccompProfile: nil, }, ImagePullSecrets: nil, Hostname: "", Subdomain: "", Affinity: nil, SchedulerName: "default-scheduler", Tolerations: [ { Key: "node.kubernetes.io/not-ready", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, { Key: "node.kubernetes.io/unreachable", Operator: "Exists", Value: "", Effect: "NoExecute", TolerationSeconds: 300, }, ], HostAliases: nil, PriorityClassName: "", Priority: 0, DNSConfig: nil, ReadinessGates: nil, RuntimeClassName: nil, EnableServiceLinks: true, PreemptionPolicy: "PreemptLowerPriority", Overhead: nil, TopologySpreadConstraints: nil, SetHostnameAsFQDN: nil, OS: nil, }, Status: { Phase: "Pending", Conditions: [ { Type: "PodScheduled", Status: "False", LastProbeTime: { Time: 0001-01-01T00:00:00Z, }, LastTransitionTime: { Time: 2022-05-23T06:03:01Z, }, Reason: "SchedulerError", Message: "running PreBind plugin \"VolumeBinding\": binding volumes: timed out waiting for the condition", }, ], Message: "", Reason: "", NominatedNodeName: "", HostIP: "", PodIP: "", PodIPs: nil, StartTime: nil, InitContainerStatuses: nil, ContainerStatuses: nil, QOSClass: "BestEffort", EphemeralContainerStatuses: nil, }, } May 23 06:08:01.955: FAIL: Unexpected error: <*pod.timeoutError | 0xc002375570>: { msg: "timed out while waiting for pod snapshotting-4352/restored-pvc-tester-s29bt to be running", } timed out while waiting for pod snapshotting-4352/restored-pvc-tester-s29bt to be running occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2() test/e2e/storage/testsuites/snapshottable.go:410 +0x1685 k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f [AfterEach] volume snapshot controller test/e2e/storage/testsuites/snapshottable.go:172 May 23 06:08:02.052: INFO: Pod restored-pvc-tester-s29bt has the following logs: May 23 06:08:02.052: INFO: Deleting pod "restored-pvc-tester-s29bt" in namespace "snapshotting-4352" May 23 06:08:02.162: INFO: Wait up to 5m0s for pod "restored-pvc-tester-s29bt" to be fully deleted May 23 06:08:02.254: INFO: deleting claim "snapshotting-4352"/"pvc-sp4rr" May 23 06:08:02.357: INFO: deleting snapshot "snapshotting-4352"/"snapshot-nzh7f" May 23 06:08:02.397: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-63e96e76-2002-44ad-844f-de8bab20386c readyToUse:false] May 23 06:08:02.397: INFO: snapshotContentName snapcontent-63e96e76-2002-44ad-844f-de8bab20386c May 23 06:08:02.481: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c to be deleted May 23 06:08:02.523: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:04.562: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:06.604: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:08.645: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:10.688: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:12.727: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:14.768: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:16.807: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:18.847: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:20.888: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:22.929: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:24.976: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:27.019: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:29.059: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:31.099: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:33.139: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:35.179: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:37.223: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:39.263: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:41.305: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:43.348: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:45.389: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:47.434: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:49.477: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:51.517: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:53.558: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:55.598: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:57.637: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:08:59.678: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:01.718: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:03.757: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:05.797: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:07.899: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:09.941: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:11.981: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:14.021: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:16.101: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:18.141: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:20.182: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:22.224: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:24.265: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:26.305: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:28.348: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:30.392: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:32.436: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:34.488: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:36.531: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:38.581: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:40.628: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:42.672: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:44.716: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:46.760: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:48.806: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:50.845: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:52.885: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:54.928: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:56.970: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:09:59.016: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:01.060: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:03.107: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:05.148: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:07.188: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:09.232: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:11.276: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:13.318: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:15.357: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:17.400: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:19.444: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:21.488: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:23.532: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:25.572: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:27.665: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:29.708: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:31.748: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:33.790: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:35.829: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:37.877: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:39.924: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:41.968: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:44.011: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:46.055: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:48.096: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:50.136: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:52.177: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:54.220: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:56.268: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:10:58.316: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:00.360: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:02.400: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:04.444: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:06.490: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:08.532: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:10.574: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:12.615: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:14.657: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:16.716: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:18.760: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:20.800: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:22.844: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:24.886: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:26.928: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:28.972: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:31.012: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:33.052: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:35.098: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:37.140: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:39.181: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:41.224: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:43.267: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:45.312: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:47.365: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:49.412: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:51.461: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:53.503: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:55.544: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:57.588: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:11:59.628: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:01.672: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:03.715: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:05.760: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:07.820: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:09.863: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:11.908: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:13.952: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:15.996: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:18.039: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:20.084: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:22.132: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:24.181: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:26.224: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:28.264: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:30.306: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:32.348: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:34.392: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:36.435: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:38.520: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:40.564: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:42.608: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:44.652: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:46.692: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:48.736: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:50.780: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:52.821: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:54.872: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:56.916: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:12:58.959: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:13:00.998: INFO: volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c has been found and is not deleted May 23 06:13:03.000: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 06:13:03.000: FAIL: Unexpected error: <*errors.errorString | 0xc0042041b0>: { s: "volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c is not deleted within 5m0s", } volumesnapshotcontents snapcontent-63e96e76-2002-44ad-844f-de8bab20386c is not deleted within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/framework.(*SnapshotResource).CleanupResource(0xc0004ba8f0, 0xc00104d980) test/e2e/storage/framework/snapshot_resource.go:204 +0xefd k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2.2() test/e2e/storage/testsuites/snapshottable.go:361 +0x28 k8s.io/kubernetes/test/e2e/storage/utils.TryFunc(0xc000493b20?) test/e2e/storage/utils/utils.go:714 +0x6d k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.2() test/e2e/storage/testsuites/snapshottable.go:167 +0x38 k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.3() test/e2e/storage/testsuites/snapshottable.go:173 +0x1b k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: Deleting pvc May 23 06:13:03.156: INFO: Deleting PersistentVolumeClaim "pd.csi.storage.gke.iodbrmg" May 23 06:13:03.256: INFO: Waiting up to 5m0s for PersistentVolume pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d to get deleted May 23 06:13:03.356: INFO: PersistentVolume pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d found and phase=Released (100.032538ms) May 23 06:13:08.404: INFO: PersistentVolume pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d was removed �[1mSTEP�[0m: Deleting sc �[1mSTEP�[0m: deleting the test namespace: snapshotting-4352 �[1mSTEP�[0m: Collecting events from namespace "snapshotting-4352". �[1mSTEP�[0m: Found 21 events. May 23 06:13:08.607: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for pd.csi.storage.gke.iodbrmg: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator May 23 06:13:08.607: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for pd.csi.storage.gke.iodbrmg: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding May 23 06:13:08.607: INFO: At 2022-05-23 05:47:17 +0000 UTC - event for pd.csi.storage.gke.iodbrmg: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-vtst_68af8cd7-0ba0-4de3-b7ea-26e9cb590ea5 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-4352/pd.csi.storage.gke.iodbrmg" May 23 06:13:08.607: INFO: At 2022-05-23 05:47:20 +0000 UTC - event for pd.csi.storage.gke.iodbrmg: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-vtst_68af8cd7-0ba0-4de3-b7ea-26e9cb590ea5 } ProvisioningSucceeded: Successfully provisioned volume pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d May 23 06:13:08.607: INFO: At 2022-05-23 05:47:21 +0000 UTC - event for pvc-snapshottable-tester-65j79: {default-scheduler } Scheduled: Successfully assigned snapshotting-4352/pvc-snapshottable-tester-65j79 to bootstrap-e2e-minion-group-qjt1 May 23 06:13:08.607: INFO: At 2022-05-23 05:47:30 +0000 UTC - event for pvc-snapshottable-tester-65j79: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d" May 23 06:13:08.607: INFO: At 2022-05-23 05:47:32 +0000 UTC - event for pvc-snapshottable-tester-65j79: {kubelet bootstrap-e2e-minion-group-qjt1} Pulled: Container image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" already present on machine May 23 06:13:08.607: INFO: At 2022-05-23 05:47:32 +0000 UTC - event for pvc-snapshottable-tester-65j79: {kubelet bootstrap-e2e-minion-group-qjt1} Started: Started container volume-tester May 23 06:13:08.607: INFO: At 2022-05-23 05:47:32 +0000 UTC - event for pvc-snapshottable-tester-65j79: {kubelet bootstrap-e2e-minion-group-qjt1} Created: Created container volume-tester May 23 06:13:08.607: INFO: At 2022-05-23 05:47:45 +0000 UTC - event for snapshot-nzh7f: {snapshot-controller } CreatingSnapshot: Waiting for a snapshot snapshotting-4352/snapshot-nzh7f to be created by the CSI driver. May 23 06:13:08.607: INFO: At 2022-05-23 05:52:47 +0000 UTC - event for pvc-snapshottable-data-tester-lllc9: {default-scheduler } Scheduled: Successfully assigned snapshotting-4352/pvc-snapshottable-data-tester-lllc9 to bootstrap-e2e-minion-group-qjt1 May 23 06:13:08.607: INFO: At 2022-05-23 05:52:56 +0000 UTC - event for pvc-snapshottable-data-tester-lllc9: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-56ddc8e1-ae2c-44d1-a1d6-c667c4cb372d" May 23 06:13:08.607: INFO: At 2022-05-23 05:52:58 +0000 UTC - event for pvc-snapshottable-data-tester-lllc9: {kubelet bootstrap-e2e-minion-group-qjt1} Started: Started container volume-tester May 23 06:13:08.607: INFO: At 2022-05-23 05:52:58 +0000 UTC - event for pvc-snapshottable-data-tester-lllc9: {kubelet bootstrap-e2e-minion-group-qjt1} Created: Created container volume-tester May 23 06:13:08.607: INFO: At 2022-05-23 05:52:58 +0000 UTC - event for pvc-snapshottable-data-tester-lllc9: {kubelet bootstrap-e2e-minion-group-qjt1} Pulled: Container image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" already present on machine May 23 06:13:08.607: INFO: At 2022-05-23 05:53:01 +0000 UTC - event for pvc-sp4rr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator May 23 06:13:08.607: INFO: At 2022-05-23 05:53:01 +0000 UTC - event for pvc-sp4rr: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-vtst_68af8cd7-0ba0-4de3-b7ea-26e9cb590ea5 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-4352/pvc-sp4rr" May 23 06:13:08.607: INFO: At 2022-05-23 05:53:01 +0000 UTC - event for pvc-sp4rr: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-vtst_68af8cd7-0ba0-4de3-b7ea-26e9cb590ea5 } ProvisioningFailed: failed to provision volume with StorageClass "snapshotting-435248nqz": error getting handle for DataSource Type VolumeSnapshot by Name snapshot-nzh7f: error getting snapshot snapshot-nzh7f from api server: the server could not find the requested resource (get volumesnapshots.snapshot.storage.k8s.io snapshot-nzh7f) May 23 06:13:08.607: INFO: At 2022-05-23 05:53:01 +0000 UTC - event for pvc-sp4rr: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding May 23 06:13:08.607: INFO: At 2022-05-23 06:03:01 +0000 UTC - event for restored-pvc-tester-s29bt: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition May 23 06:13:08.607: INFO: At 2022-05-23 06:08:02 +0000 UTC - event for restored-pvc-tester-s29bt: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "restored-pvc-tester-s29bt" not found May 23 06:13:08.702: INFO: POD NODE PHASE GRACE CONDITIONS May 23 06:13:08.702: INFO: May 23 06:13:08.804: INFO: Logging node info for node bootstrap-e2e-master May 23 06:13:08.904: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 5486 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-master"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:08:10 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:08:10 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:08:10 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:08:10 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:13:08.904: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 06:13:09.008: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 06:13:09.127: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.127: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 06:13:09.127: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.127: INFO: Container kube-scheduler ready: true, restart count 0 May 23 06:13:09.127: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.127: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 06:13:09.127: INFO: csi-gce-pd-node-mgnhf started at 2022-05-23 05:47:04 +0000 UTC (0+2 container statuses recorded) May 23 06:13:09.127: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 06:13:09.127: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 06:13:09.127: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 06:13:09.127: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:13:09.127: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:13:09.127: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.128: INFO: Container etcd-container ready: true, restart count 0 May 23 06:13:09.128: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.128: INFO: Container kube-apiserver ready: true, restart count 0 May 23 06:13:09.128: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.128: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 06:13:09.128: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.128: INFO: Container etcd-container ready: true, restart count 0 May 23 06:13:09.128: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.128: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 06:13:09.495: INFO: Latency metrics for node bootstrap-e2e-master May 23 06:13:09.496: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 06:13:09.556: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 6114 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-minion-group-qjt1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:52:56 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:53:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:08:28 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:08:28 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:08:28 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:08:28 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:13:09.556: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 06:13:09.656: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 06:13:09.805: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.805: INFO: Container autoscaler ready: true, restart count 0 May 23 06:13:09.805: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.805: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:13:09.805: INFO: csi-gce-pd-node-zpx49 started at 2022-05-23 05:47:04 +0000 UTC (0+2 container statuses recorded) May 23 06:13:09.805: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 06:13:09.805: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 06:13:09.805: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.805: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:13:09.805: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 06:13:09.805: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:13:09.805: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:13:09.805: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:13:09.805: INFO: Container coredns ready: true, restart count 0 May 23 06:13:10.163: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 06:13:10.163: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 06:13:10.256: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 6284 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-minion-group-tls5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:11:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:11:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:11:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:11:54 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:13:10.256: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 06:13:10.360: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 06:13:10.479: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:13:10.479: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:13:10.479: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:13:10.479: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:13:10.479: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:13:10.479: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 06:13:10.479: INFO: Container coredns ready: true, restart count 0 May 23 06:13:10.479: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 06:13:10.479: INFO: Container metrics-server ready: true, restart count 0 May 23 06:13:10.479: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 06:13:10.479: INFO: csi-gce-pd-node-9l5lp started at 2022-05-23 05:47:04 +0000 UTC (0+2 container statuses recorded) May 23 06:13:10.479: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 06:13:10.479: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 06:13:10.479: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:13:10.479: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:13:10.920: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 06:13:10.920: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 06:13:10.960: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 6285 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-minion-group-vtst"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:45:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:11:52 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:11:52 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:11:52 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:11:52 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:52 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:52 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:11:52 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:12:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:13:10.978: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 06:13:11.060: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 06:13:11.223: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:13:11.223: INFO: Container default-http-backend ready: true, restart count 0 May 23 06:13:11.223: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:13:11.223: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:13:11.223: INFO: csi-gce-pd-controller-0 started at 2022-05-23 05:47:04 +0000 UTC (0+5 container statuses recorded) May 23 06:13:11.223: INFO: Container csi-attacher ready: true, restart count 0 May 23 06:13:11.223: INFO: Container csi-provisioner ready: true, restart count 0 May 23 06:13:11.223: INFO: Container csi-resizer ready: true, restart count 0 May 23 06:13:11.223: INFO: Container csi-snapshotter ready: true, restart count 0 May 23 06:13:11.223: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 06:13:11.223: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:13:11.223: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:13:11.223: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:13:11.223: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:13:11.223: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:13:11.223: INFO: csi-gce-pd-node-trztk started at 2022-05-23 05:47:04 +0000 UTC (0+2 container statuses recorded) May 23 06:13:11.223: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 06:13:11.223: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 06:13:11.223: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:13:11.223: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 06:13:11.538: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst �[1mSTEP�[0m: Waiting for namespaces [snapshotting-4352] to vanish May 23 06:18:11.802: INFO: error deleting namespace snapshotting-4352: timed out waiting for the condition �[1mSTEP�[0m: uninstalling csi gce-pd driver May 23 06:18:11.802: INFO: deleting *v1.ServiceAccount: snapshotting-4352-2783/csi-attacher May 23 06:18:11.907: INFO: deleting *v1.ClusterRole: external-attacher-runner-snapshotting-4352 May 23 06:18:12.007: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-4352 May 23 06:18:12.107: INFO: deleting *v1.Role: snapshotting-4352-2783/external-attacher-cfg-snapshotting-4352 May 23 06:18:12.205: INFO: deleting *v1.RoleBinding: snapshotting-4352-2783/csi-attacher-role-cfg May 23 06:18:12.305: INFO: deleting *v1.ServiceAccount: snapshotting-4352-2783/csi-provisioner May 23 06:18:12.404: INFO: deleting *v1.ClusterRole: external-provisioner-runner-snapshotting-4352 May 23 06:18:12.504: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-4352 May 23 06:18:12.603: INFO: deleting *v1.Role: snapshotting-4352-2783/external-provisioner-cfg-snapshotting-4352 May 23 06:18:12.703: INFO: deleting *v1.RoleBinding: snapshotting-4352-2783/csi-provisioner-role-cfg May 23 06:18:12.803: INFO: deleting *v1.ServiceAccount: snapshotting-4352-2783/csi-gce-pd-controller-sa May 23 06:18:12.909: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-4352 May 23 06:18:13.004: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-4352 May 23 06:18:13.104: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-4352 May 23 06:18:13.203: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-4352 May 23 06:18:13.304: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-4352 May 23 06:18:13.403: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-4352 May 23 06:18:13.503: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-4352 May 23 06:18:13.608: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-4352 May 23 06:18:13.703: INFO: deleting *v1.Role: snapshotting-4352-2783/csi-gce-pd-leaderelection-role-snapshotting-4352 May 23 06:18:13.803: INFO: deleting *v1.RoleBinding: snapshotting-4352-2783/csi-gce-pd-controller-leaderelection-binding May 23 06:18:13.903: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-4352 May 23 06:18:14.003: INFO: deleting *v1.DaemonSet: snapshotting-4352-2783/csi-gce-pd-node May 23 06:18:14.106: INFO: deleting *v1.StatefulSet: snapshotting-4352-2783/csi-gce-pd-controller �[1mSTEP�[0m: deleting the driver namespace: snapshotting-4352-2783 �[1mSTEP�[0m: Collecting events from namespace "snapshotting-4352-2783". �[1mSTEP�[0m: Found 76 events. May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-controller: {statefulset-controller } SuccessfulCreate: create Pod csi-gce-pd-controller-0 in StatefulSet csi-gce-pd-controller successful May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-controller-0: {default-scheduler } Scheduled: Successfully assigned snapshotting-4352-2783/csi-gce-pd-controller-0 to bootstrap-e2e-minion-group-vtst May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-mgnhf May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-trztk May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-9l5lp May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-zpx49 May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node-9l5lp: {default-scheduler } Scheduled: Successfully assigned snapshotting-4352-2783/csi-gce-pd-node-9l5lp to bootstrap-e2e-minion-group-tls5 May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node-mgnhf: {default-scheduler } Scheduled: Successfully assigned snapshotting-4352-2783/csi-gce-pd-node-mgnhf to bootstrap-e2e-master May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node-trztk: {default-scheduler } Scheduled: Successfully assigned snapshotting-4352-2783/csi-gce-pd-node-trztk to bootstrap-e2e-minion-group-vtst May 23 06:18:14.323: INFO: At 2022-05-23 05:47:04 +0000 UTC - event for csi-gce-pd-node-zpx49: {default-scheduler } Scheduled: Successfully assigned snapshotting-4352-2783/csi-gce-pd-node-zpx49 to bootstrap-e2e-minion-group-qjt1 May 23 06:18:14.323: INFO: At 2022-05-23 05:47:05 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:05 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 1.12054811s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:05 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:05 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:05 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Created: Created container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Started: Started container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 1.924463643s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 964.333272ms May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Created: Created container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:06 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 1.177929808s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-snapshotter May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-snapshotter May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" in 1.613416101s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Created: Created container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Started: Started container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:07 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:08 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:08 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Started: Started container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 05:47:10 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0" in 2.44628372s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:10 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 4.049815626s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:11 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-provisioner May 23 06:18:14.323: INFO: At 2022-05-23 05:47:11 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-provisioner May 23 06:18:14.323: INFO: At 2022-05-23 05:47:11 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:11 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Started: Started container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:11 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Created: Created container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:11 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 4.181710694s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:12 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Started: Started container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:12 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Created: Created container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:13 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 6.138060136s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:14 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-attacher May 23 06:18:14.323: INFO: At 2022-05-23 05:47:14 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-attacher May 23 06:18:14.323: INFO: At 2022-05-23 05:47:14 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0" May 23 06:18:14.323: INFO: At 2022-05-23 05:47:14 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0" in 2.637063767s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:14 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 6.547015165s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:14 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:14 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:15 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Started: Started container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:15 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Created: Created container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-resizer May 23 06:18:14.323: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-resizer May 23 06:18:14.323: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0" in 1.474786062s May 23 06:18:14.323: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine May 23 06:18:14.323: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 05:47:16 +0000 UTC - event for external-snapshotter-leader-pd-csi-storage-gke-io: {external-snapshotter-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 06:18:14.323: INFO: At 2022-05-23 05:47:17 +0000 UTC - event for external-attacher-leader-pd-csi-storage-gke-io: {external-attacher-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 06:18:14.323: INFO: At 2022-05-23 05:47:17 +0000 UTC - event for external-resizer-pd-csi-storage-gke-io: {external-resizer-pd-csi-storage-gke-io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 06:18:14.323: INFO: At 2022-05-23 05:47:17 +0000 UTC - event for pd-csi-storage-gke-io: {pd.csi.storage.gke.io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-9l5lp_snapshotting-4352-2783(82a188c7-d4d8-4261-b01d-d37cba7090a5)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "818eecfd64a85ec608b06b4b9503a496a742b49a12fe5475b59ef0443f021daf": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: "" May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Killing: Stopping container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-9l5lp: {kubelet bootstrap-e2e-minion-group-tls5} Killing: Stopping container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Killing: Stopping container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-mgnhf: {kubelet bootstrap-e2e-master} Killing: Stopping container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-trztk_snapshotting-4352-2783(bd3f7351-33be-4d73-9dc0-61ebc9a30c37)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "4e74786b38d6ba652c0ac109e6af132fe8ba9975bf332351ecfe9f9aa28a80ea": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: "" May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Killing: Stopping container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-trztk: {kubelet bootstrap-e2e-minion-group-vtst} Killing: Stopping container gce-pd-driver May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Killing: Stopping container csi-driver-registrar May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-zpx49_snapshotting-4352-2783(b6795584-0069-4a09-af43-91c8efe9ec9a)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "97a789cc31f3680c2ac7507d74d80862f831182780dbba3eb4415933068f847f": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: "" May 23 06:18:14.323: INFO: At 2022-05-23 06:18:14 +0000 UTC - event for csi-gce-pd-node-zpx49: {kubelet bootstrap-e2e-minion-group-qjt1} Killing: Stopping container gce-pd-driver May 23 06:18:14.457: INFO: POD NODE PHASE GRACE CONDITIONS May 23 06:18:14.457: INFO: csi-gce-pd-controller-0 bootstrap-e2e-minion-group-vtst Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:16 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:16 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC }] May 23 06:18:14.457: INFO: csi-gce-pd-node-9l5lp bootstrap-e2e-minion-group-tls5 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:12 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:12 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC }] May 23 06:18:14.457: INFO: csi-gce-pd-node-mgnhf bootstrap-e2e-master Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:15 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:15 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC }] May 23 06:18:14.457: INFO: csi-gce-pd-node-trztk bootstrap-e2e-minion-group-vtst Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:14 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:14 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC }] May 23 06:18:14.457: INFO: csi-gce-pd-node-zpx49 bootstrap-e2e-minion-group-qjt1 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:13 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:13 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 05:47:04 +0000 UTC }] May 23 06:18:14.457: INFO: May 23 06:18:15.198: INFO: Unable to fetch snapshotting-4352-2783/csi-gce-pd-node-zpx49/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-zpx49) May 23 06:18:15.236: INFO: Unable to fetch snapshotting-4352-2783/csi-gce-pd-node-zpx49/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-zpx49) May 23 06:18:15.277: INFO: Logging node info for node bootstrap-e2e-master May 23 06:18:15.316: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 7221 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:16 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:15.317: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 06:18:15.357: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 06:18:15.406: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container etcd-container ready: true, restart count 0 May 23 06:18:15.406: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container kube-apiserver ready: true, restart count 0 May 23 06:18:15.406: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 06:18:15.406: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container etcd-container ready: true, restart count 0 May 23 06:18:15.406: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 06:18:15.406: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 06:18:15.406: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:15.406: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:15.406: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 06:18:15.406: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container kube-scheduler ready: true, restart count 0 May 23 06:18:15.406: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.406: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 06:18:15.406: INFO: csi-gce-pd-node-mgnhf started at 2022-05-23 05:47:04 +0000 UTC (0+2 container statuses recorded) May 23 06:18:15.406: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 06:18:15.406: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 06:18:15.633: INFO: Latency metrics for node bootstrap-e2e-master May 23 06:18:15.633: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 06:18:15.700: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 7230 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:52:56 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:53:09 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:15.700: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 06:18:15.764: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 06:18:15.841: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.841: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:18:15.841: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.841: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:18:15.841: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 06:18:15.841: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:15.841: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:15.841: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.841: INFO: Container coredns ready: true, restart count 0 May 23 06:18:15.841: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:15.841: INFO: Container autoscaler ready: true, restart count 0 May 23 06:18:16.026: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 06:18:16.026: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 06:18:16.066: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 7216 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:16.067: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 06:18:16.112: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 06:18:16.166: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:18:16.166: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:18:16.166: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:18:16.166: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:16.166: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:16.166: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:16.166: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:18:16.166: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 06:18:16.166: INFO: Container coredns ready: true, restart count 0 May 23 06:18:16.166: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 06:18:16.166: INFO: Container metrics-server ready: true, restart count 0 May 23 06:18:16.166: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 06:18:16.344: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 06:18:16.344: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 06:18:16.384: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 7219 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:45:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:16.384: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 06:18:16.424: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 06:18:16.470: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:16.470: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 06:18:16.470: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:16.470: INFO: Container default-http-backend ready: true, restart count 0 May 23 06:18:16.470: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:16.470: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:18:16.470: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:18:16.470: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:18:16.470: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:18:16.470: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:16.470: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:16.640: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst �[1mSTEP�[0m: Waiting for namespaces [snapshotting-4352-2783] to vanish [AfterEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/framework/framework.go:187 �[1mSTEP�[0m: Collecting events from namespace "snapshotting-4352". �[1mSTEP�[0m: Found 0 events. May 23 06:18:22.797: INFO: POD NODE PHASE GRACE CONDITIONS May 23 06:18:22.797: INFO: May 23 06:18:22.837: INFO: Logging node info for node bootstrap-e2e-master May 23 06:18:22.877: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 7221 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:16 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:13:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:22.877: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 06:18:22.960: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 06:18:23.016: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 06:18:23.016: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container kube-scheduler ready: true, restart count 0 May 23 06:18:23.016: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 06:18:23.016: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container etcd-container ready: true, restart count 0 May 23 06:18:23.016: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container kube-apiserver ready: true, restart count 0 May 23 06:18:23.016: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 06:18:23.016: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container etcd-container ready: true, restart count 0 May 23 06:18:23.016: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.016: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 06:18:23.016: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 06:18:23.016: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:23.016: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:23.203: INFO: Latency metrics for node bootstrap-e2e-master May 23 06:18:23.203: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 06:18:23.243: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 7230 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:52:56 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:53:09 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:56 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:13:34 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:23.243: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 06:18:23.283: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 06:18:23.330: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.330: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:18:23.330: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 06:18:23.330: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:23.330: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:23.330: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.330: INFO: Container coredns ready: true, restart count 0 May 23 06:18:23.330: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.330: INFO: Container autoscaler ready: true, restart count 0 May 23 06:18:23.330: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.330: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:18:23.491: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 06:18:23.491: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 06:18:23.531: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 7216 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:23.532: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 06:18:23.572: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 06:18:23.617: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.617: INFO: Container coredns ready: true, restart count 0 May 23 06:18:23.617: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 06:18:23.617: INFO: Container metrics-server ready: true, restart count 0 May 23 06:18:23.617: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 06:18:23.617: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.617: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:18:23.617: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:18:23.617: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:23.617: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:23.617: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.617: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:18:23.775: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 06:18:23.775: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 06:18:23.814: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 7219 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:45:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 05:47:15 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 06:16:53 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 06:18:03 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 06:18:23.814: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 06:18:23.855: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 06:18:23.901: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 06:18:23.901: INFO: Container metadata-proxy ready: true, restart count 0 May 23 06:18:23.901: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 06:18:23.901: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.901: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 06:18:23.901: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.901: INFO: Container default-http-backend ready: true, restart count 0 May 23 06:18:23.901: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.901: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 06:18:23.901: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 06:18:23.901: INFO: Container kube-proxy ready: true, restart count 0 May 23 06:18:24.073: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst May 23 06:18:24.073: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready �[1mSTEP�[0m: Destroying namespace "snapshotting-4352" for this suite.
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sPre\-provisioned\sSnapshot\s\(retain\spolicy\)\]\ssnapshottable\[Feature\:VolumeSnapshotDataSource\]\svolume\ssnapshot\scontroller\s\sshould\scheck\ssnapshot\sfields\,\scheck\srestore\scorrectly\sworks\safter\smodifying\ssource\sdata\,\scheck\sdeletion\s\(persistent\)$'
test/e2e/storage/testsuites/snapshottable.go:278 May 23 07:06:44.297: Unexpected error: <*errors.errorString | 0xc0015b93c0>: { s: "VolumeSnapshot snapshot-9g98s is not ready within 5m0s", } VolumeSnapshot snapshot-9g98s is not ready within 5m0s occurred test/e2e/storage/utils/snapshot.go:86from junit_01.xml
[BeforeEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/framework/framework.go:186 �[1mSTEP�[0m: Creating a kubernetes client May 23 07:01:18.252: INFO: >>> kubeConfig: /workspace/.kube/config �[1mSTEP�[0m: Building a namespace api object, basename snapshotting �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace �[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent) test/e2e/storage/testsuites/snapshottable.go:278 �[1mSTEP�[0m: deploying csi gce-pd driver �[1mSTEP�[0m: Building a driver namespace object, basename snapshotting-9794 �[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace May 23 07:01:18.847: INFO: Found CI service account key at /etc/service-account/service-account.json May 23 07:01:18.847: INFO: Running cp [/etc/service-account/service-account.json /tmp/53c06189-8829-4992-846e-31c64b4acb44/cloud-sa.json] May 23 07:01:18.890: INFO: Shredding file /tmp/53c06189-8829-4992-846e-31c64b4acb44/cloud-sa.json May 23 07:01:18.890: INFO: Running shred [--remove /tmp/53c06189-8829-4992-846e-31c64b4acb44/cloud-sa.json] May 23 07:01:18.930: INFO: File /tmp/53c06189-8829-4992-846e-31c64b4acb44/cloud-sa.json successfully shredded May 23 07:01:18.937: INFO: creating *v1.ServiceAccount: snapshotting-9794-5143/csi-attacher May 23 07:01:18.980: INFO: creating *v1.ClusterRole: external-attacher-runner-snapshotting-9794 May 23 07:01:18.980: INFO: Define cluster role external-attacher-runner-snapshotting-9794 May 23 07:01:19.018: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-9794 May 23 07:01:19.057: INFO: creating *v1.Role: snapshotting-9794-5143/external-attacher-cfg-snapshotting-9794 May 23 07:01:19.096: INFO: creating *v1.RoleBinding: snapshotting-9794-5143/csi-attacher-role-cfg May 23 07:01:19.136: INFO: creating *v1.ServiceAccount: snapshotting-9794-5143/csi-provisioner May 23 07:01:19.176: INFO: creating *v1.ClusterRole: external-provisioner-runner-snapshotting-9794 May 23 07:01:19.176: INFO: Define cluster role external-provisioner-runner-snapshotting-9794 May 23 07:01:19.215: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-9794 May 23 07:01:19.255: INFO: creating *v1.Role: snapshotting-9794-5143/external-provisioner-cfg-snapshotting-9794 May 23 07:01:19.295: INFO: creating *v1.RoleBinding: snapshotting-9794-5143/csi-provisioner-role-cfg May 23 07:01:19.334: INFO: creating *v1.ServiceAccount: snapshotting-9794-5143/csi-gce-pd-controller-sa May 23 07:01:19.374: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-9794 May 23 07:01:19.374: INFO: Define cluster role csi-gce-pd-provisioner-role-snapshotting-9794 May 23 07:01:19.414: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-9794 May 23 07:01:19.453: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-9794 May 23 07:01:19.453: INFO: Define cluster role csi-gce-pd-attacher-role-snapshotting-9794 May 23 07:01:19.492: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-9794 May 23 07:01:19.531: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-9794 May 23 07:01:19.532: INFO: Define cluster role csi-gce-pd-resizer-role-snapshotting-9794 May 23 07:01:19.571: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-9794 May 23 07:01:19.611: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-9794 May 23 07:01:19.611: INFO: Define cluster role csi-gce-pd-snapshotter-role-snapshotting-9794 May 23 07:01:19.650: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-9794 May 23 07:01:19.689: INFO: creating *v1.Role: snapshotting-9794-5143/csi-gce-pd-leaderelection-role-snapshotting-9794 May 23 07:01:19.729: INFO: creating *v1.RoleBinding: snapshotting-9794-5143/csi-gce-pd-controller-leaderelection-binding May 23 07:01:19.769: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-9794 May 23 07:01:19.809: INFO: creating *v1.DaemonSet: snapshotting-9794-5143/csi-gce-pd-node May 23 07:01:19.852: INFO: creating *v1.StatefulSet: snapshotting-9794-5143/csi-gce-pd-controller May 23 07:01:20.017: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-qjt1 May 23 07:01:25.201: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-tls5 May 23 07:01:25.240: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-vtst May 23 07:01:25.278: INFO: Creating resource for dynamic PV May 23 07:01:25.278: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(pd.csi.storage.gke.io) supported size:{ 1Mi} �[1mSTEP�[0m: creating a StorageClass snapshotting-9794pkkwz �[1mSTEP�[0m: creating a claim May 23 07:01:25.318: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil �[1mSTEP�[0m: [init] starting a pod to use the claim �[1mSTEP�[0m: [init] check pod success May 23 07:01:25.480: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-vdn2w" in namespace "snapshotting-9794" to be "Succeeded or Failed" May 23 07:01:25.519: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 38.642653ms May 23 07:01:27.559: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 2.079226994s May 23 07:01:29.558: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 4.077876558s May 23 07:01:31.559: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 6.078558768s May 23 07:01:33.559: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 8.078335166s May 23 07:01:35.558: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 10.0781656s May 23 07:01:37.559: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 12.078675034s May 23 07:01:39.558: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 14.078074915s May 23 07:01:41.566: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Pending", Reason="", readiness=false. Elapsed: 16.085826519s May 23 07:01:43.577: INFO: Pod "pvc-snapshottable-tester-vdn2w": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.096359239s �[1mSTEP�[0m: Saw pod success May 23 07:01:43.577: INFO: Pod "pvc-snapshottable-tester-vdn2w" satisfied condition "Succeeded or Failed" �[1mSTEP�[0m: [init] checking the claim May 23 07:01:43.617: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pd.csi.storage.gke.iocw6ps] to have phase Bound May 23 07:01:43.657: INFO: PersistentVolumeClaim pd.csi.storage.gke.iocw6ps found and phase=Bound (40.459899ms) �[1mSTEP�[0m: [init] checking the PV �[1mSTEP�[0m: [init] deleting the pod May 23 07:01:43.795: INFO: Pod pvc-snapshottable-tester-vdn2w has the following logs: May 23 07:01:43.795: INFO: Deleting pod "pvc-snapshottable-tester-vdn2w" in namespace "snapshotting-9794" May 23 07:01:43.840: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-vdn2w" to be fully deleted �[1mSTEP�[0m: [init] waiting until the node=bootstrap-e2e-minion-group-qjt1 is not using the volume=projects/kubernetes-gci-es-logging/zones/us-west1-b/disks/pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb May 23 07:01:43.926: INFO: current volumes in use: [] May 23 07:01:43.926: INFO: WaitUntil finished successfully after 48.066355ms �[1mSTEP�[0m: creating a SnapshotClass �[1mSTEP�[0m: creating a dynamic VolumeSnapshot May 23 07:01:44.047: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-9g98s to become ready May 23 07:01:44.096: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:01:46.136: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:01:48.176: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:01:50.216: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:01:52.256: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:01:54.296: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:01:56.336: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:01:58.378: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:00.417: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:02.459: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:04.500: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:06.542: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:08.583: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:10.624: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:12.665: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:14.705: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:16.745: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:18.784: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:20.825: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:22.865: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:24.905: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:26.944: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:28.984: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:31.036: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:33.077: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:35.116: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:37.157: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:39.197: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:41.240: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:43.281: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:45.368: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:47.408: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:49.448: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:51.488: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:53.530: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:55.572: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:57.616: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:02:59.660: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:01.704: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:03.748: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:05.821: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:07.865: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:09.908: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:11.948: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:13.988: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:16.033: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:18.073: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:20.113: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:22.152: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:24.191: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:26.235: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:28.275: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:30.316: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:32.364: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:34.403: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:36.442: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:38.481: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:40.522: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:42.561: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:44.601: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:46.645: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:48.684: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:50.723: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:52.763: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:54.805: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:56.845: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:03:58.884: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:00.924: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:02.988: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:05.035: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:07.075: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:09.116: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:11.156: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:13.196: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:15.236: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:17.277: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:19.317: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:21.357: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:23.398: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:25.471: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:27.511: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:29.551: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:31.590: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:33.631: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:35.671: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:37.711: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:39.752: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:41.793: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:43.832: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:45.882: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:47.921: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:49.964: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:52.003: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:54.043: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:56.083: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:04:58.123: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:00.163: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:02.203: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:04.242: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:06.283: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:08.323: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:10.362: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:12.403: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:14.444: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:16.483: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:18.564: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:20.605: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:22.647: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:24.687: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:26.727: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:28.768: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:30.809: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:32.849: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:34.889: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:36.929: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:38.969: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:41.011: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:43.051: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:45.090: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:47.131: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:49.171: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:51.211: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:53.250: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:55.291: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:57.334: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:05:59.374: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:01.426: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:03.467: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:05.560: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:07.606: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:09.651: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:11.691: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:13.731: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:15.771: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:17.812: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:19.852: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:21.892: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:23.937: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:25.978: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:28.018: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:30.057: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:32.096: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:34.136: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:36.175: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:38.215: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:40.255: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:42.296: INFO: VolumeSnapshot snapshot-9g98s found but is not ready. May 23 07:06:44.296: INFO: WaitUntil failed after reaching the timeout 5m0s May 23 07:06:44.297: FAIL: Unexpected error: <*errors.errorString | 0xc0015b93c0>: { s: "VolumeSnapshot snapshot-9g98s is not ready within 5m0s", } VolumeSnapshot snapshot-9g98s is not ready within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eae40, 0xc004898488?}, 0xc004482a40) test/e2e/storage/utils/snapshot.go:86 +0x1ad k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f09442f6580, 0xc00118aea0}, 0xc002e57200, {{0x71f693a, 0x28}, {0x0, 0x0}, {0x7133cd2, 0x9}, {0x0, ...}, ...}, ...) test/e2e/storage/framework/snapshot_resource.go:92 +0x246 k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2() test/e2e/storage/testsuites/snapshottable.go:359 +0xb3d k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: updating the snapshot content deletion policy to retain [AfterEach] volume snapshot controller test/e2e/storage/testsuites/snapshottable.go:172 �[1mSTEP�[0m: Deleting pvc May 23 07:06:44.374: INFO: Deleting PersistentVolumeClaim "pd.csi.storage.gke.iocw6ps" May 23 07:06:44.419: INFO: Waiting up to 5m0s for PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb to get deleted May 23 07:06:44.457: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (38.045187ms) May 23 07:06:49.496: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (5.076956871s) May 23 07:06:54.535: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (10.116132306s) May 23 07:06:59.578: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (15.158863436s) May 23 07:07:04.621: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (20.201418187s) May 23 07:07:09.659: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (25.240158228s) May 23 07:07:14.701: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (30.282198323s) May 23 07:07:19.743: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (35.324216696s) May 23 07:07:24.786: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (40.366838479s) May 23 07:07:29.827: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (45.408298205s) May 23 07:07:34.871: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (50.451489988s) May 23 07:07:39.910: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (55.491162117s) May 23 07:07:44.950: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m0.530511737s) May 23 07:07:49.988: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m5.569254642s) May 23 07:07:55.035: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m10.616286386s) May 23 07:08:00.079: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m15.659609206s) May 23 07:08:05.118: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m20.699034168s) May 23 07:08:10.161: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m25.741781675s) May 23 07:08:15.201: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m30.782200816s) May 23 07:08:20.240: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m35.821313281s) May 23 07:08:25.280: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m40.860661425s) May 23 07:08:30.322: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m45.903282493s) May 23 07:08:35.361: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m50.942009016s) May 23 07:08:40.400: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (1m55.980581176s) May 23 07:08:45.443: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m1.023464537s) May 23 07:08:50.483: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m6.063488346s) May 23 07:08:55.523: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m11.103723383s) May 23 07:09:00.566: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m16.14666954s) May 23 07:09:05.626: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m21.207277552s) May 23 07:09:10.666: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m26.246641281s) May 23 07:09:15.771: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m31.351776784s) May 23 07:09:20.810: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m36.391220138s) May 23 07:09:25.858: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m41.439084479s) May 23 07:09:30.898: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m46.478856381s) May 23 07:09:35.944: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m51.525329181s) May 23 07:09:40.984: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (2m56.565262367s) May 23 07:09:46.024: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m1.605080125s) May 23 07:09:51.064: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m6.645092405s) May 23 07:09:56.107: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m11.688121355s) May 23 07:10:01.146: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m16.726968112s) May 23 07:10:06.219: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m21.79975421s) May 23 07:10:11.319: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m26.899541772s) May 23 07:10:16.368: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m31.949172209s) May 23 07:10:21.418: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m36.999296426s) May 23 07:10:26.469: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m42.049900195s) May 23 07:10:31.520: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m47.101019574s) May 23 07:10:36.569: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m52.150006504s) May 23 07:10:41.618: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (3m57.198673178s) May 23 07:10:46.668: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m2.249354755s) May 23 07:10:51.718: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m7.299002292s) May 23 07:10:56.769: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m12.350141372s) May 23 07:11:01.818: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m17.39897084s) May 23 07:11:06.869: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m22.450313513s) May 23 07:11:11.933: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m27.513554089s) May 23 07:11:17.020: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m32.600791453s) May 23 07:11:22.080: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m37.661179646s) May 23 07:11:27.121: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m42.701480899s) May 23 07:11:32.221: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m47.8022039s) May 23 07:11:37.264: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m52.844489882s) May 23 07:11:42.319: INFO: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb found and phase=Bound (4m57.900318874s) �[1mSTEP�[0m: Deleting sc May 23 07:11:47.372: FAIL: Unexpected error: <errors.aggregate | len:1, cap:1>: [ { msg: "persistent Volume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb not deleted by dynamic provisioner: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb still exists within 5m0s", err: { s: "PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb still exists within 5m0s", }, }, ] persistent Volume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb not deleted by dynamic provisioner: PersistentVolume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb still exists within 5m0s occurred Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.1.1() test/e2e/storage/testsuites/snapshottable.go:142 +0x3c k8s.io/kubernetes/test/e2e/storage/utils.TryFunc(0xc0003270c0?) test/e2e/storage/utils/utils.go:714 +0x6d k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.2() test/e2e/storage/testsuites/snapshottable.go:167 +0x38 k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.3() test/e2e/storage/testsuites/snapshottable.go:173 +0x1b k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?) test/e2e/e2e.go:130 +0x686 k8s.io/kubernetes/test/e2e.TestE2E(0x1?) test/e2e/e2e_test.go:136 +0x19 testing.tRunner(0xc00025d1e0, 0x73d5cc0) /usr/local/go/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1486 +0x35f �[1mSTEP�[0m: deleting the test namespace: snapshotting-9794 �[1mSTEP�[0m: Collecting events from namespace "snapshotting-9794". �[1mSTEP�[0m: Found 10 events. May 23 07:11:47.470: INFO: At 2022-05-23 07:01:25 +0000 UTC - event for pd.csi.storage.gke.iocw6ps: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding May 23 07:11:47.470: INFO: At 2022-05-23 07:01:25 +0000 UTC - event for pd.csi.storage.gke.iocw6ps: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator May 23 07:11:47.470: INFO: At 2022-05-23 07:01:25 +0000 UTC - event for pd.csi.storage.gke.iocw6ps: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-vtst_0efa2090-4063-45fd-9c7e-f5590c4fc8b3 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-9794/pd.csi.storage.gke.iocw6ps" May 23 07:11:47.470: INFO: At 2022-05-23 07:01:29 +0000 UTC - event for pd.csi.storage.gke.iocw6ps: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-vtst_0efa2090-4063-45fd-9c7e-f5590c4fc8b3 } ProvisioningSucceeded: Successfully provisioned volume pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb May 23 07:11:47.470: INFO: At 2022-05-23 07:01:29 +0000 UTC - event for pvc-snapshottable-tester-vdn2w: {default-scheduler } Scheduled: Successfully assigned snapshotting-9794/pvc-snapshottable-tester-vdn2w to bootstrap-e2e-minion-group-qjt1 May 23 07:11:47.470: INFO: At 2022-05-23 07:01:38 +0000 UTC - event for pvc-snapshottable-tester-vdn2w: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-80c6c4f2-4dfa-46b4-b861-c468624d2bbb" May 23 07:11:47.470: INFO: At 2022-05-23 07:01:40 +0000 UTC - event for pvc-snapshottable-tester-vdn2w: {kubelet bootstrap-e2e-minion-group-qjt1} Pulled: Container image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" already present on machine May 23 07:11:47.470: INFO: At 2022-05-23 07:01:40 +0000 UTC - event for pvc-snapshottable-tester-vdn2w: {kubelet bootstrap-e2e-minion-group-qjt1} Created: Created container volume-tester May 23 07:11:47.470: INFO: At 2022-05-23 07:01:40 +0000 UTC - event for pvc-snapshottable-tester-vdn2w: {kubelet bootstrap-e2e-minion-group-qjt1} Started: Started container volume-tester May 23 07:11:47.470: INFO: At 2022-05-23 07:01:44 +0000 UTC - event for snapshot-9g98s: {snapshot-controller } CreatingSnapshot: Waiting for a snapshot snapshotting-9794/snapshot-9g98s to be created by the CSI driver. May 23 07:11:47.569: INFO: POD NODE PHASE GRACE CONDITIONS May 23 07:11:47.569: INFO: May 23 07:11:47.671: INFO: Logging node info for node bootstrap-e2e-master May 23 07:11:47.770: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 16482 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-master"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:11:47.770: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 07:11:47.869: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 07:11:47.988: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container etcd-container ready: true, restart count 0 May 23 07:11:47.988: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container kube-apiserver ready: true, restart count 0 May 23 07:11:47.988: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 07:11:47.988: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container etcd-container ready: true, restart count 0 May 23 07:11:47.988: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 07:11:47.988: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 07:11:47.988: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:11:47.988: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:11:47.988: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 07:11:47.988: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container kube-scheduler ready: true, restart count 0 May 23 07:11:47.988: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:11:47.988: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 07:11:47.988: INFO: csi-gce-pd-node-wpzjf started at 2022-05-23 07:01:20 +0000 UTC (0+2 container statuses recorded) May 23 07:11:47.988: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 07:11:47.988: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 07:11:48.326: INFO: Latency metrics for node bootstrap-e2e-master May 23 07:11:48.326: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 07:11:48.421: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 16080 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-minion-group-qjt1"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 07:01:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:06:48 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:06:48 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:06:48 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:06:48 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:11:48.422: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 07:11:48.518: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 07:11:48.658: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 07:11:48.658: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:11:48.658: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 07:11:48.658: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:11:48.658: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:11:48.658: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:11:48.658: INFO: Container coredns ready: true, restart count 0 May 23 07:11:48.658: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:11:48.658: INFO: Container autoscaler ready: true, restart count 0 May 23 07:11:48.658: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:11:48.658: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:11:48.658: INFO: csi-gce-pd-node-6kd7b started at 2022-05-23 07:01:19 +0000 UTC (0+2 container statuses recorded) May 23 07:11:48.658: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 07:11:48.658: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 07:11:48.968: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 07:11:48.968: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 07:11:49.020: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 16438 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-minion-group-tls5"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:07:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:11:49.021: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 07:11:49.118: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 07:11:49.265: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 07:11:49.266: INFO: Container coredns ready: true, restart count 0 May 23 07:11:49.266: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 07:11:49.266: INFO: Container metrics-server ready: true, restart count 0 May 23 07:11:49.266: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 07:11:49.266: INFO: csi-gce-pd-node-62z5m started at 2022-05-23 07:01:20 +0000 UTC (0+2 container statuses recorded) May 23 07:11:49.266: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 07:11:49.266: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 07:11:49.266: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:11:49.266: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:11:49.266: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:11:49.266: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:11:49.266: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:11:49.266: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:11:49.266: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:11:49.571: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 07:11:49.571: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 07:11:49.620: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 16797 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/kubernetes-gci-es-logging/zones/us-west1-b/instances/bootstrap-e2e-minion-group-vtst"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 07:00:18 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:06:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:06:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:06:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:06:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:06:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:06:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:06:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:11:49.620: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 07:11:49.718: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 07:11:49.842: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:11:49.843: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:11:49.843: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:11:49.843: INFO: csi-gce-pd-node-gvjb4 started at 2022-05-23 07:01:20 +0000 UTC (0+2 container statuses recorded) May 23 07:11:49.843: INFO: Container csi-driver-registrar ready: true, restart count 0 May 23 07:11:49.843: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 07:11:49.843: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:11:49.843: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 07:11:49.843: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:11:49.843: INFO: Container default-http-backend ready: true, restart count 0 May 23 07:11:49.843: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:11:49.843: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:11:49.843: INFO: csi-gce-pd-controller-0 started at 2022-05-23 07:01:20 +0000 UTC (0+5 container statuses recorded) May 23 07:11:49.843: INFO: Container csi-attacher ready: true, restart count 0 May 23 07:11:49.843: INFO: Container csi-provisioner ready: true, restart count 0 May 23 07:11:49.843: INFO: Container csi-resizer ready: true, restart count 0 May 23 07:11:49.843: INFO: Container csi-snapshotter ready: true, restart count 0 May 23 07:11:49.843: INFO: Container gce-pd-driver ready: true, restart count 0 May 23 07:11:49.843: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:11:49.843: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:11:50.191: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst �[1mSTEP�[0m: Waiting for namespaces [snapshotting-9794] to vanish �[1mSTEP�[0m: uninstalling csi gce-pd driver May 23 07:12:02.337: INFO: deleting *v1.ServiceAccount: snapshotting-9794-5143/csi-attacher May 23 07:12:02.426: INFO: deleting *v1.ClusterRole: external-attacher-runner-snapshotting-9794 May 23 07:12:02.528: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-9794 May 23 07:12:02.626: INFO: deleting *v1.Role: snapshotting-9794-5143/external-attacher-cfg-snapshotting-9794 May 23 07:12:02.734: INFO: deleting *v1.RoleBinding: snapshotting-9794-5143/csi-attacher-role-cfg May 23 07:12:02.823: INFO: deleting *v1.ServiceAccount: snapshotting-9794-5143/csi-provisioner May 23 07:12:02.936: INFO: deleting *v1.ClusterRole: external-provisioner-runner-snapshotting-9794 May 23 07:12:03.055: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-9794 May 23 07:12:03.331: INFO: deleting *v1.Role: snapshotting-9794-5143/external-provisioner-cfg-snapshotting-9794 May 23 07:12:03.372: INFO: deleting *v1.RoleBinding: snapshotting-9794-5143/csi-provisioner-role-cfg May 23 07:12:03.412: INFO: deleting *v1.ServiceAccount: snapshotting-9794-5143/csi-gce-pd-controller-sa May 23 07:12:03.458: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-9794 May 23 07:12:03.520: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-9794 May 23 07:12:03.621: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-9794 May 23 07:12:03.720: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-9794 May 23 07:12:03.820: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-9794 May 23 07:12:03.920: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-9794 May 23 07:12:04.021: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-9794 May 23 07:12:04.120: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-9794 May 23 07:12:04.220: INFO: deleting *v1.Role: snapshotting-9794-5143/csi-gce-pd-leaderelection-role-snapshotting-9794 May 23 07:12:04.320: INFO: deleting *v1.RoleBinding: snapshotting-9794-5143/csi-gce-pd-controller-leaderelection-binding May 23 07:12:04.420: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-9794 May 23 07:12:04.521: INFO: deleting *v1.DaemonSet: snapshotting-9794-5143/csi-gce-pd-node May 23 07:12:04.621: INFO: deleting *v1.StatefulSet: snapshotting-9794-5143/csi-gce-pd-controller �[1mSTEP�[0m: deleting the driver namespace: snapshotting-9794-5143 �[1mSTEP�[0m: Collecting events from namespace "snapshotting-9794-5143". �[1mSTEP�[0m: Found 69 events. May 23 07:12:04.822: INFO: At 2022-05-23 07:01:19 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-6kd7b May 23 07:12:04.822: INFO: At 2022-05-23 07:01:19 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-gvjb4 May 23 07:12:04.822: INFO: At 2022-05-23 07:01:19 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-62z5m May 23 07:12:04.822: INFO: At 2022-05-23 07:01:19 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-wpzjf May 23 07:12:04.822: INFO: At 2022-05-23 07:01:19 +0000 UTC - event for csi-gce-pd-node-6kd7b: {default-scheduler } Scheduled: Successfully assigned snapshotting-9794-5143/csi-gce-pd-node-6kd7b to bootstrap-e2e-minion-group-qjt1 May 23 07:12:04.822: INFO: At 2022-05-23 07:01:20 +0000 UTC - event for csi-gce-pd-controller: {statefulset-controller } SuccessfulCreate: create Pod csi-gce-pd-controller-0 in StatefulSet csi-gce-pd-controller successful May 23 07:12:04.822: INFO: At 2022-05-23 07:01:20 +0000 UTC - event for csi-gce-pd-controller-0: {default-scheduler } Scheduled: Successfully assigned snapshotting-9794-5143/csi-gce-pd-controller-0 to bootstrap-e2e-minion-group-vtst May 23 07:12:04.822: INFO: At 2022-05-23 07:01:20 +0000 UTC - event for csi-gce-pd-node-62z5m: {default-scheduler } Scheduled: Successfully assigned snapshotting-9794-5143/csi-gce-pd-node-62z5m to bootstrap-e2e-minion-group-tls5 May 23 07:12:04.822: INFO: At 2022-05-23 07:01:20 +0000 UTC - event for csi-gce-pd-node-gvjb4: {default-scheduler } Scheduled: Successfully assigned snapshotting-9794-5143/csi-gce-pd-node-gvjb4 to bootstrap-e2e-minion-group-vtst May 23 07:12:04.822: INFO: At 2022-05-23 07:01:20 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Created: Created container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:20 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:20 +0000 UTC - event for csi-gce-pd-node-wpzjf: {default-scheduler } Scheduled: Successfully assigned snapshotting-9794-5143/csi-gce-pd-node-wpzjf to bootstrap-e2e-master May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-kmjgp" : failed to sync configmap cache: timed out waiting for the condition May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-zr8hl" : failed to sync configmap cache: timed out waiting for the condition May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Created: Created container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Started: Started container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Started: Started container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Created: Created container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-4872b" : failed to sync configmap cache: timed out waiting for the condition May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Started: Started container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Started: Started container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:21 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Created: Created container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-provisioner May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-snapshotter May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-snapshotter May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" in 249.959528ms May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Container image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Container image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-provisioner May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Created: Created container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Started: Started container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Started: Started container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Created: Created container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:22 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Container image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-attacher May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-attacher May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Started: Started container csi-resizer May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Created: Created container csi-resizer May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for external-snapshotter-leader-pd-csi-storage-gke-io: {external-snapshotter-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 07:12:04.822: INFO: At 2022-05-23 07:01:23 +0000 UTC - event for pd-csi-storage-gke-io: {pd.csi.storage.gke.io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 07:12:04.822: INFO: At 2022-05-23 07:01:24 +0000 UTC - event for external-attacher-leader-pd-csi-storage-gke-io: {external-attacher-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 07:12:04.822: INFO: At 2022-05-23 07:01:24 +0000 UTC - event for external-resizer-pd-csi-storage-gke-io: {external-resizer-pd-csi-storage-gke-io/bootstrap-e2e-minion-group-vtst } LeaderElection: bootstrap-e2e-minion-group-vtst became leader May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-vtst} Killing: Stopping container csi-snapshotter May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Killing: Stopping container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} Killing: Stopping container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-62z5m: {kubelet bootstrap-e2e-minion-group-tls5} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-62z5m_snapshotting-9794-5143(7e1a1984-ddc0-47f1-a85d-cd7f146de1c8)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "39c7c78bf6a926e0d0d5996008810dc51daf0882f8cab8f0676b412448adbd07": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: "" May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Killing: Stopping container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-6kd7b_snapshotting-9794-5143(cf37b2ed-66b4-448d-8b16-4686ce06a978)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "07e8076b30b00f3985cd839809ba962dc31009dcde163721bc5318675c7d138f": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: "" May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-6kd7b: {kubelet bootstrap-e2e-minion-group-qjt1} Killing: Stopping container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-gvjb4_snapshotting-9794-5143(1a077f2b-c624-4790-8c2c-9ddc83e1209d)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "2ef00f4df862004d971d6d5948bccada0b07254e4ecdf9b24b7ebaa4c53bc03b": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: "" May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Killing: Stopping container gce-pd-driver May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-gvjb4: {kubelet bootstrap-e2e-minion-group-vtst} Killing: Stopping container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Killing: Stopping container csi-driver-registrar May 23 07:12:04.822: INFO: At 2022-05-23 07:12:04 +0000 UTC - event for csi-gce-pd-node-wpzjf: {kubelet bootstrap-e2e-master} Killing: Stopping container gce-pd-driver May 23 07:12:04.872: INFO: POD NODE PHASE GRACE CONDITIONS May 23 07:12:04.872: INFO: csi-gce-pd-controller-0 bootstrap-e2e-minion-group-vtst Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:20 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:24 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:24 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:20 +0000 UTC }] May 23 07:12:04.872: INFO: csi-gce-pd-node-62z5m bootstrap-e2e-minion-group-tls5 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:20 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:23 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:23 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:19 +0000 UTC }] May 23 07:12:04.872: INFO: csi-gce-pd-node-6kd7b bootstrap-e2e-minion-group-qjt1 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:19 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:22 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:22 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:19 +0000 UTC }] May 23 07:12:04.873: INFO: csi-gce-pd-node-gvjb4 bootstrap-e2e-minion-group-vtst Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:20 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:23 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:23 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:19 +0000 UTC }] May 23 07:12:04.873: INFO: csi-gce-pd-node-wpzjf bootstrap-e2e-master Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:20 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:22 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:22 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-23 07:01:20 +0000 UTC }] May 23 07:12:04.873: INFO: May 23 07:12:14.923: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-controller-0/csi-snapshotter logs: the server could not find the requested resource (get pods csi-gce-pd-controller-0) May 23 07:12:14.962: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-controller-0/csi-provisioner logs: the server could not find the requested resource (get pods csi-gce-pd-controller-0) May 23 07:12:15.000: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-controller-0/csi-attacher logs: the server could not find the requested resource (get pods csi-gce-pd-controller-0) May 23 07:12:15.039: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-controller-0/csi-resizer logs: the server could not find the requested resource (get pods csi-gce-pd-controller-0) May 23 07:12:15.077: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-controller-0/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-controller-0) May 23 07:12:15.115: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-62z5m/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-62z5m) May 23 07:12:15.163: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-62z5m/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-62z5m) May 23 07:12:15.201: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-6kd7b/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-6kd7b) May 23 07:12:15.239: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-6kd7b/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-6kd7b) May 23 07:12:15.277: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-gvjb4/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-gvjb4) May 23 07:12:15.315: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-gvjb4/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-gvjb4) May 23 07:12:15.353: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-wpzjf/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-wpzjf) May 23 07:12:15.391: INFO: Unable to fetch snapshotting-9794-5143/csi-gce-pd-node-wpzjf/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-wpzjf) May 23 07:12:15.432: INFO: Logging node info for node bootstrap-e2e-master May 23 07:12:15.471: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 87879e0b-37b3-4c92-bae5-62c237dc75dc 17004 0 2022-05-23 05:41:48 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-23 05:41:48 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:09:24 +0000 UTC,LastTransitionTime:2022-05-23 05:42:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.84.103,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ac2ef3acd00e10a40c2094f1784cf5a1,SystemUUID:ac2ef3ac-d00e-10a4-0c20-94f1784cf5a1,BootID:55aa17fc-691c-4b50-84b7-1f6d6625e37f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:117128515,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:50936642,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:12:15.472: INFO: Logging kubelet events for node bootstrap-e2e-master May 23 07:12:15.510: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master May 23 07:12:15.559: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container kube-apiserver ready: true, restart count 0 May 23 07:12:15.559: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container kube-controller-manager ready: true, restart count 2 May 23 07:12:15.559: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container etcd-container ready: true, restart count 0 May 23 07:12:15.559: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container l7-lb-controller ready: true, restart count 3 May 23 07:12:15.559: INFO: metadata-proxy-v0.1-d2bkx started at 2022-05-23 05:42:03 +0000 UTC (0+2 container statuses recorded) May 23 07:12:15.559: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:12:15.559: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:12:15.559: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container etcd-container ready: true, restart count 0 May 23 07:12:15.559: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container kube-scheduler ready: true, restart count 0 May 23 07:12:15.559: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-23 05:41:11 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container kube-addon-manager ready: true, restart count 0 May 23 07:12:15.559: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-23 05:40:36 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.559: INFO: Container konnectivity-server-container ready: true, restart count 0 May 23 07:12:15.778: INFO: Latency metrics for node bootstrap-e2e-master May 23 07:12:15.778: INFO: Logging node info for node bootstrap-e2e-minion-group-qjt1 May 23 07:12:15.819: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qjt1 83824386-d134-4288-ab75-69e6549a7e09 16981 0 2022-05-23 05:41:45 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qjt1 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-qjt1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 07:01:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-qjt1,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:12:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:12:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:12:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:12:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:12:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:12:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:12:01 +0000 UTC,LastTransitionTime:2022-05-23 05:41:50 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:45 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:11:55 +0000 UTC,LastTransitionTime:2022-05-23 05:41:56 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.83.141.185,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qjt1.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d9a6ea47aeaaa51dd55d1137c9830583,SystemUUID:d9a6ea47-aeaa-a51d-d55d-1137c9830583,BootID:8a69ae25-9dd6-4d22-803f-4584596dea0f,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:12:15.819: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qjt1 May 23 07:12:15.859: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qjt1 May 23 07:12:15.909: INFO: kube-proxy-bootstrap-e2e-minion-group-qjt1 started at 2022-05-23 05:41:45 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.909: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:12:15.909: INFO: metadata-proxy-v0.1-6ccjv started at 2022-05-23 05:41:46 +0000 UTC (0+2 container statuses recorded) May 23 07:12:15.909: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:12:15.909: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:12:15.909: INFO: coredns-77c6689697-h8bvd started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.909: INFO: Container coredns ready: true, restart count 0 May 23 07:12:15.909: INFO: kube-dns-autoscaler-569c995844-xdgzt started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.909: INFO: Container autoscaler ready: true, restart count 0 May 23 07:12:15.909: INFO: konnectivity-agent-5kj79 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:12:15.909: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:12:16.097: INFO: Latency metrics for node bootstrap-e2e-minion-group-qjt1 May 23 07:12:16.097: INFO: Logging node info for node bootstrap-e2e-minion-group-tls5 May 23 07:12:16.140: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tls5 4f8600e5-20a7-46fb-afbc-004cdbef4fb2 16983 0 2022-05-23 05:41:46 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tls5 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-tls5 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:52 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 05:42:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:23 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-tls5,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:12:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:12:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:12:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:12:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:12:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:12:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:12:00 +0000 UTC,LastTransitionTime:2022-05-23 05:41:51 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:09:08 +0000 UTC,LastTransitionTime:2022-05-23 05:41:57 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.105.11.196,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tls5.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:357f7d2688e9565f2ab8d2a0181e509f,SystemUUID:357f7d26-88e9-565f-2ab8-d2a0181e509f,BootID:54e6cb33-9317-4b55-ad86-929aaa96a082,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:12:16.141: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tls5 May 23 07:12:16.179: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tls5 May 23 07:12:16.226: INFO: coredns-77c6689697-s6hdv started at 2022-05-23 05:42:21 +0000 UTC (0+1 container statuses recorded) May 23 07:12:16.226: INFO: Container coredns ready: true, restart count 0 May 23 07:12:16.226: INFO: metrics-server-v0.5.2-fcb7b948c-dcm95 started at 2022-05-23 05:42:30 +0000 UTC (0+2 container statuses recorded) May 23 07:12:16.226: INFO: Container metrics-server ready: true, restart count 0 May 23 07:12:16.226: INFO: Container metrics-server-nanny ready: true, restart count 0 May 23 07:12:16.226: INFO: kube-proxy-bootstrap-e2e-minion-group-tls5 started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:12:16.226: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:12:16.226: INFO: metadata-proxy-v0.1-pnbtb started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:12:16.226: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:12:16.226: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:12:16.226: INFO: konnectivity-agent-sstjw started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:12:16.226: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:12:16.385: INFO: Latency metrics for node bootstrap-e2e-minion-group-tls5 May 23 07:12:16.385: INFO: Logging node info for node bootstrap-e2e-minion-group-vtst May 23 07:12:16.425: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-vtst 2875750a-e539-4475-8f8d-09a32d18f752 16994 0 2022-05-23 05:41:47 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-vtst kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-vtst topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-23 05:41:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-05-23 05:41:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-05-23 07:00:18 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-23 07:01:24 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://kubernetes-gci-es-logging/us-west1-b/bootstrap-e2e-minion-group-vtst,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-23 07:11:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-23 07:11:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-23 07:11:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-23 07:11:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-23 07:11:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-23 07:11:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-23 07:11:57 +0000 UTC,LastTransitionTime:2022-05-23 05:41:48 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-23 05:42:17 +0000 UTC,LastTransitionTime:2022-05-23 05:42:17 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-23 07:11:16 +0000 UTC,LastTransitionTime:2022-05-23 05:41:47 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:104.196.244.149,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-vtst.c.kubernetes-gci-es-logging.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b7a0975b7b7657233b5144ea4aab2f22,SystemUUID:b7a0975b-7b76-5723-3b51-44ea4aab2f22,BootID:9f0391d4-e296-4546-9526-e34e1b7e64cd,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.587+fbb57172799b1b,KubeProxyVersion:v1.25.0-alpha.0.587+fbb57172799b1b,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:112030526,},ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.587_fbb57172799b1b],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 23 07:12:16.425: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-vtst May 23 07:12:16.463: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-vtst May 23 07:12:16.510: INFO: konnectivity-agent-wjczj started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:12:16.510: INFO: Container konnectivity-agent ready: true, restart count 0 May 23 07:12:16.510: INFO: kube-proxy-bootstrap-e2e-minion-group-vtst started at 2022-05-23 05:41:47 +0000 UTC (0+1 container statuses recorded) May 23 07:12:16.510: INFO: Container kube-proxy ready: true, restart count 0 May 23 07:12:16.510: INFO: metadata-proxy-v0.1-vtmvp started at 2022-05-23 05:41:47 +0000 UTC (0+2 container statuses recorded) May 23 07:12:16.510: INFO: Container metadata-proxy ready: true, restart count 0 May 23 07:12:16.510: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 May 23 07:12:16.510: INFO: volume-snapshot-controller-0 started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:12:16.510: INFO: Container volume-snapshot-controller ready: true, restart count 0 May 23 07:12:16.510: INFO: l7-default-backend-5df94ffdcb-klrqr started at 2022-05-23 05:42:17 +0000 UTC (0+1 container statuses recorded) May 23 07:12:16.510: INFO: Container default-http-backend ready: true, restart count 0 May 23 07:12:16.681: INFO: Latency metrics for node bootstrap-e2e-minion-group-vtst �[1mSTEP�[0m: Waiting for namespaces [snapshotting-9794-5143] to vanish [AfterEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/framework/framework.go:187 May 23 07:12:22.762: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
Filter through log files | View test history on testgrid
error during ./hack/ginkgo-e2e.sh --ginkgo.focus=\[Feature:VolumeSnapshotDataSource\] --ginkgo.skip=\[Disruptive\]|\[Flaky\] --minStartupPods=8 --report-dir=/workspace/_artifacts --disable-log-dump=true (interrupted): exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest --timeout triggered
from junit_runner.xml
Filter through log files | View test history on testgrid
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Ephemeral Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works, check deletion (ephemeral)
kubetest Check APIReachability
kubetest Deferred TearDown
kubetest DumpClusterLogs
kubetest Extract
kubetest GetDeployer
kubetest IsUp
kubetest Prepare
kubetest TearDown
kubetest TearDown Previous
kubetest Up
kubetest kubectl version
kubetest list nodes
kubetest listResources Before
kubetest listResources Down
kubetest listResources Up
kubetest test setup
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] patching/updating a mutating webhook should work [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should honor timeout [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate configmap [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with different stored version [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with pruning [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert a non homogeneous list of CRs [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourceDefinition Watch [Privileged:ClusterAdmin] CustomResourceDefinition Watch watch on custom resource definition objects [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] custom resource defaulting for requests and from storage works [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] should include custom resource definition resources in discovery documents [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] [Flaky] kubectl explain works for CR with the same resource name as built-in object.
Kubernetes e2e suite [sig-api-machinery] Etcd failure [Disruptive] should recover from SIGKILL
Kubernetes e2e suite [sig-api-machinery] Garbage collector should delete RS created by deployment when not orphaning [Conformance]
Kubernetes e2e suite [sig-api-machinery] Garbage collector should orphan RS created by deployment when deleteOptions.PropagationPolicy is Orphan [Conformance]
Kubernetes e2e suite [sig-api-machinery] Namespaces [Serial] should always delete fast (ALL of 100 namespaces in 150 seconds) [Feature:ComprehensiveNamespaceDraining]
Kubernetes e2e suite [sig-api-machinery] Namespaces [Serial] should delete fast enough (90 percent of 100 namespaces in 150 seconds)
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's multiple priority class scope (quota set to pod count: 2) against 2 pods with same priority classes.
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's priority class scope (cpu, memory quota set) against a pod with same priority class.
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's priority class scope (quota set to pod count: 1) against a pod with different priority class (ScopeSelectorOpNotIn).
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's priority class scope (quota set to pod count: 1) against a pod with same priority class.
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should be able to update and delete ResourceQuota. [Conformance]
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a persistent volume claim with a storage class
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a service. [Conformance]
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should create a ResourceQuota and ensure its status is promptly calculated. [Conformance]
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should verify ResourceQuota with best effort scope. [Conformance]
Kubernetes e2e suite [sig-api-machinery] Server request timeout default timeout should be used if the specified timeout in the request URL is 0s
Kubernetes e2e suite [sig-api-machinery] ServerSideApply should create an applied object if it does not already exist
Kubernetes e2e suite [sig-api-machinery] ServerSideApply should give up ownership of a field if forced applied by a controller
Kubernetes e2e suite [sig-api-machinery] ServerSideApply should not remove a field if an owner unsets the field but other managers still have ownership of the field
Kubernetes e2e suite [sig-api-machinery] Servers with support for Table transformation should return generic metadata details across all namespaces for nodes
Kubernetes e2e suite [sig-api-machinery] Watchers should be able to restart watching from the last resource version observed by the previous watch [Conformance]
Kubernetes e2e suite [sig-api-machinery] Watchers should receive events on concurrent watches in same order [Conformance]
Kubernetes e2e suite [sig-api-machinery] client-go should negotiate watch and report errors with accept "application/vnd.kubernetes.protobuf"
Kubernetes e2e suite [sig-api-machinery] client-go should negotiate watch and report errors with accept "application/vnd.kubernetes.protobuf,application/json"
Kubernetes e2e suite [sig-apps] CronJob should delete failed finished jobs with limit of one job
Kubernetes e2e suite [sig-apps] CronJob should delete successful finished jobs with limit of one successful job
Kubernetes e2e suite [sig-apps] Daemon set [Serial] should not update pod when spec was updated and update strategy is OnDelete
Kubernetes e2e suite [sig-apps] Daemon set [Serial] should run and stop simple daemon [Conformance]
Kubernetes e2e suite [sig-apps] Daemon set [Serial] should verify changes to a daemon set status [Conformance]
Kubernetes e2e suite [sig-apps] Deployment Deployment should have a working scale subresource [Conformance]
Kubernetes e2e suite [sig-apps] Deployment should run the lifecycle of a Deployment [Conformance]
Kubernetes e2e suite [sig-apps] DisruptionController should block an eviction until the PDB is updated to allow it [Conformance]
Kubernetes e2e suite [sig-apps] DisruptionController should create a PodDisruptionBudget [Conformance]
Kubernetes e2e suite [sig-apps] Job should manage the lifecycle of a job [Conformance]
Kubernetes e2e suite [sig-apps] Job should remove pods when job is deleted
Kubernetes e2e suite [sig-apps] ReplicationController should serve a basic image on each replica with a private image
Kubernetes e2e suite [sig-apps] ReplicationController should test the lifecycle of a ReplicationController [Conformance]
Kubernetes e2e suite [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance]
Kubernetes e2e suite [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Should recreate evicted statefulset [Conformance]
Kubernetes e2e suite [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should not deadlock when a pod's predecessor fails
Kubernetes e2e suite [sig-apps] stateful Upgrade [Feature:StatefulUpgrade] stateful upgrade should maintain a functioning cluster
Kubernetes e2e suite [sig-auth] Certificates API [Privileged:ClusterAdmin] should support CSR API operations [Conformance]
Kubernetes e2e suite [sig-auth] ServiceAccounts ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance]
Kubernetes e2e suite [sig-auth] ServiceAccounts should run through the lifecycle of a ServiceAccount [Conformance]
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthenticator] The kubelet's main port 10250 should reject requests with no credentials
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthorizer] A node shouldn't be able to delete another node
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthorizer] Getting an existing configmap should exit with the Forbidden error
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthorizer] Getting an existing secret should exit with the Forbidden error
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaler scalability [Slow] should scale up at all [Feature:ClusterAutoscalerScalability1]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] should add node to the particular mig [Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] should be able to scale down by draining multiple pods one by one as dictated by pdb[Feature:ClusterSizeAutoscalingScaleDown]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] should correctly scale down after a node is not needed [Feature:ClusterSizeAutoscalingScaleDown]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] should correctly scale down after a node is not needed and one node is broken [Feature:ClusterSizeAutoscalingScaleDown]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] should increase cluster size if pod requesting EmptyDir volume is pending [Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] shouldn't be able to scale down when rescheduling a pod is required, but pdb doesn't allow drain[Feature:ClusterSizeAutoscalingScaleDown]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] shouldn't increase cluster size if pending pod is too large [Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] shouldn't scale down when non expendable pod is running [Feature:ClusterSizeAutoscalingScaleDown]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] shouldn't scale up when expendable pod is preempted [Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) [Serial] [Slow] Deployment Should scale from 1 pod to 3 pods and from 3 to 5
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) [Serial] [Slow] Deployment Should scale from 5 pods to 3 pods and from 3 to 1
Kubernetes e2e suite [sig-autoscaling] [HPA] Horizontal pod autoscaling (scale resource: Custom Metrics from Stackdriver) should scale down with Custom Metric of type Object from Stackdriver [Feature:CustomMetricsAutoscaling]
Kubernetes e2e suite [sig-autoscaling] [HPA] Horizontal pod autoscaling (scale resource: Custom Metrics from Stackdriver) should scale down with Custom Metric of type Pod from Stackdriver with Prometheus [Feature:CustomMetricsAutoscaling]
Kubernetes e2e suite [sig-cli] Kubectl Port forwarding With a server listening on 0.0.0.0 that expects a client request should support a client that connects, sends DATA, and disconnects
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl cluster-info should check if Kubernetes control plane services is included in cluster-info [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl copy should copy a file from a running Pod
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl create quota should create a quota with scopes
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl create quota should reject quota with invalid scopes
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl expose should create services for rc [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl server-side dry-run should check if kubectl can dry-run update Pods [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl taint [Serial] should remove all the taints with the same key off a node
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl validation should create/apply a valid CR with arbitrary-extra properties for CRD with partially-specified validation schema
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl validation should detect unknown metadata fields in both the root and embedded object of a CR
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl version should check is all data is printed [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should contain last line of the log
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should return command exit codes running a failing command
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should support exec
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should support exec through kubectl proxy
Kubernetes e2e suite [sig-cloud-provider-gcp] Downgrade [Feature:Downgrade] cluster downgrade should maintain a functioning cluster [Feature:ClusterDowngrade]
Kubernetes e2e suite [sig-cloud-provider-gcp] GKE node pools [Feature:GKENodePool] should create a cluster with multiple node pools [Feature:GKENodePool]
Kubernetes e2e suite [sig-cloud-provider-gcp] Nodes [Disruptive] Resize [Slow] should be able to add nodes
Kubernetes e2e suite [sig-cloud-provider-gcp] Reboot [Disruptive] [Feature:Reboot] each node by ordering clean reboot and ensure they function upon restart
Kubernetes e2e suite [sig-cloud-provider-gcp] Reboot [Disruptive] [Feature:Reboot] each node by switching off the network interface and ensure they function upon switch on
Kubernetes e2e suite [sig-cloud-provider-gcp] Recreate [Feature:Recreate] recreate nodes and ensure they function upon restart
Kubernetes e2e suite [sig-cloud-provider-gcp] Restart [Disruptive] should restart all nodes and ensure all nodes and pods recover
Kubernetes e2e suite [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] cluster upgrade should maintain a functioning cluster [Feature:ClusterUpgrade]
Kubernetes e2e suite [sig-instrumentation] Events API should delete a collection of events [Conformance]
Kubernetes e2e suite [sig-instrumentation] Events should ensure that an event can be fetched, patched, deleted, and listed [Conformance]
Kubernetes e2e suite [sig-instrumentation] MetricsGrabber should grab all metrics from a Kubelet.
Kubernetes e2e suite [sig-instrumentation] Stackdriver Monitoring should have cluster metrics [Feature:StackdriverMonitoring]
Kubernetes e2e suite [sig-instrumentation] Stackdriver Monitoring should run Custom Metrics - Stackdriver Adapter for new resource model [Feature:StackdriverCustomMetrics]
Kubernetes e2e suite [sig-network] Conntrack should be able to preserve UDP traffic when server pod cycles for a ClusterIP service
Kubernetes e2e suite [sig-network] DNS configMap nameserver Change stubDomain should be able to change stubDomain configuration [Slow][Serial]
Kubernetes e2e suite [sig-network] DNS configMap nameserver Forward PTR lookup should forward PTR records lookup to upstream nameserver [Slow][Serial]
Kubernetes e2e suite [sig-network] DNS should provide /etc/hosts entries for the cluster [Conformance]
Kubernetes e2e suite [sig-network] DNS should provide DNS for pods for Subdomain [Conformance]
Kubernetes e2e suite [sig-network] DNS should provide DNS for services [Conformance]
Kubernetes e2e suite [sig-network] DNS should provide DNS for the cluster [Conformance]
Kubernetes e2e suite [sig-network] DNS should resolve DNS of partial qualified names for services [LinuxOnly] [Conformance]
Kubernetes e2e suite [sig-network] DNS should resolve DNS of partial qualified names for the cluster [LinuxOnly]
Kubernetes e2e suite [sig-network] Firewall rule [Slow] [Serial] should create valid firewall rules for LoadBalancer type service
Kubernetes e2e suite [sig-network] IngressClass API should support creating IngressClass API operations [Conformance]
Kubernetes e2e suite [sig-network] LoadBalancers ESIPP [Slow] should handle updates to ExternalTrafficPolicy field
Kubernetes e2e suite [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints
Kubernetes e2e suite [sig-network] LoadBalancers ESIPP [Slow] should work for type=LoadBalancer
Kubernetes e2e suite [sig-network] LoadBalancers should be able to create LoadBalancer Service without NodePort and change it [Slow]
Kubernetes e2e suite [sig-network] LoadBalancers should handle load balancer cleanup finalizer for service [Slow]
Kubernetes e2e suite [sig-network] LoadBalancers should only allow access from service loadbalancer source ranges [Slow]
Kubernetes e2e suite [sig-network] LoadBalancers should reconcile LB health check interval [Slow][Serial][Disruptive]
Kubernetes e2e suite [sig-network] Loadbalancing: L7 GCE [Slow] [Feature:Ingress] should conform to Ingress spec
Kubernetes e2e suite [sig-network] Loadbalancing: L7 GCE [Slow] [Feature:NEG] [Flaky] should be able to switch between IG and NEG modes
Kubernetes e2e suite [sig-network] Loadbalancing: L7 GCE [Slow] [Feature:NEG] [Flaky] should sync endpoints for both Ingress-referenced NEG and standalone NEG
Kubernetes e2e suite [sig-network] Loadbalancing: L7 GCE [Slow] [Feature:NEG] [Flaky] should sync endpoints to NEG
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should allow egress access on one named port [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should allow ingress access from namespace on one named port [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should allow ingress access on one named port [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should deny ingress access to updated pod [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should deny ingress from pods on other namespaces [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce egress policy allowing traffic to a server in a different namespace based on PodSelector and NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy based on NamespaceSelector with MatchExpressions using default ns label [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy based on PodSelector and NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy based on Ports [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy to allow traffic from pods within server namespace based on PodSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy to allow traffic only from a different namespace, based on NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should ensure an IP overlapping both IPBlock.CIDR and IPBlock.Except is allowed [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should not allow access by TCP when a policy specifies only UDP [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should stop enforcing policies after they are deleted [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should support allow-all policy [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should support denying of egress traffic on the client side (even if the server explicitly allows this traffic) [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol [LinuxOnly] NetworkPolicy between server and client using UDP should enforce policy based on Ports [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol [LinuxOnly] NetworkPolicy between server and client using UDP should enforce policy to allow traffic only from a pod in a different namespace based on PodSelector and NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicy [Feature:SCTPConnectivity][LinuxOnly][Disruptive] NetworkPolicy between server and client using SCTP should enforce policy based on Ports [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicy [Feature:SCTPConnectivity][LinuxOnly][Disruptive] NetworkPolicy between server and client using SCTP should enforce policy to allow traffic only from a pod in a different namespace based on PodSelector and NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce multiple egress policies with egress allow-all policy taking precedence [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce multiple ingress policies with ingress allow-all policy taking precedence [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce multiple, stacked policies with overlapping podSelectors [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce policy based on PodSelector or NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce policy based on Ports [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce policy to allow traffic from pods within server namespace based on PodSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should ensure an IP overlapping both IPBlock.CIDR and IPBlock.Except is allowed [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Pods should function for node-pod communication: sctp [LinuxOnly][Feature:SCTPConnectivity][Disruptive]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for client IP based session affinity: http [LinuxOnly]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for endpoint-Service: http
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for endpoint-Service: sctp [Feature:SCTPConnectivity][Disruptive]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for node-Service: sctp [Feature:SCTPConnectivity][Disruptive]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for pod-Service: udp
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should support basic nodePort: udp functionality
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should update nodePort: http [Slow]
Kubernetes e2e suite [sig-network] Networking IPerf2 [Feature:Networking-Performance] should run iperf2
Kubernetes e2e suite [sig-network] Networking should provide Internet connection for containers [Feature:Networking-IPv6][Experimental][LinuxOnly]
Kubernetes e2e suite [sig-network] SCTP [LinuxOnly] should allow creating a basic SCTP service with pod and endpoints
Kubernetes e2e suite [sig-network] Services should be rejected when no endpoints exist
Kubernetes e2e suite [sig-network] Services should check NodePort out-of-range
Kubernetes e2e suite [sig-network] Services should fallback to terminating endpoints when there are no ready endpoints with internalTrafficPolicy=Cluster [Feature:ProxyTerminatingEndpoints]
Kubernetes e2e suite [sig-network] Services should find a service from listing all namespaces [Conformance]
Kubernetes e2e suite [sig-network] Services should release NodePorts on delete
Kubernetes e2e suite [sig-network] Services should respect internalTrafficPolicy=Local Pod (hostNetwork: true) to Pod [Feature:ServiceInternalTrafficPolicy]
Kubernetes e2e suite [sig-network] Services should serve a basic endpoint from pods [Conformance]
Kubernetes e2e suite [sig-network] Services should serve multiport endpoints from pods [Conformance]
Kubernetes e2e suite [sig-network] Services should work after restarting apiserver [Disruptive]
Kubernetes e2e suite [sig-network] Services should work after restarting kube-proxy [Disruptive]
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] Granular Checks: Services Secondary IP Family [LinuxOnly] should be able to handle large requests: udp
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] Granular Checks: Services Secondary IP Family [LinuxOnly] should function for client IP based session affinity: udp [LinuxOnly]
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] Granular Checks: Services Secondary IP Family [LinuxOnly] should function for endpoint-Service: http
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] should create pod, add ipv6 and ipv4 ip to pod ips
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] should create service with ipv6,v4 cluster ip
Kubernetes e2e suite [sig-network] [Feature:Topology Hints] should distribute endpoints evenly
Kubernetes e2e suite [sig-node] ConfigMap should run through a ConfigMap lifecycle [Conformance]
Kubernetes e2e suite [sig-node] Container Lifecycle Hook when create a pod with lifecycle hook should execute poststart exec hook properly [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Container Lifecycle Hook when create a pod with lifecycle hook should execute poststart http hook properly [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test on terminated container should report termination message as empty when pod succeeds and TerminationMessagePolicy FallbackToLogsOnError is set [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test on terminated container should report termination message from file when pod succeeds and TerminationMessagePolicy FallbackToLogsOnError is set [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Downward API [Serial] [Disruptive] [NodeFeature:DownwardAPIHugePages] Downward API tests for hugepages should provide default limits.hugepages-<pagesize> from node allocatable
Kubernetes e2e suite [sig-node] Kubelet [Serial] [Slow] regular resource usage tracking [Feature:RegularResourceUsageTracking] resource tracking for 100 pods per node
Kubernetes e2e suite [sig-node] Kubelet when scheduling a busybox command that always fails in a pod should be possible to delete [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Kubelet when scheduling a busybox command that always fails in a pod should have an terminated reason [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Mount propagation should propagate mounts within defined scopes
Kubernetes e2e suite [sig-node] NodeLease NodeLease the kubelet should create and update a lease in the kube-node-lease namespace
Kubernetes e2e suite [sig-node] Pods Extended Pods Set QOS Class should be set on Pods with matching resource requests and limits for memory and cpu [Conformance]
Kubernetes e2e suite [sig-node] Pods should support pod readiness gates [NodeConformance]
Kubernetes e2e suite [sig-node] PreStop graceful pod terminated should wait until preStop hook completes the process
Kubernetes e2e suite [sig-node] Probing container should *not* be restarted with a /healthz http liveness probe [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Probing container should *not* be restarted with a exec "cat /tmp/health" liveness probe [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Probing container should *not* be restarted with a tcp:8080 liveness probe [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Probing container should be restarted startup probe fails
Kubernetes e2e suite [sig-node] Probing container should be restarted with a failing exec liveness probe that took longer than the timeout
Kubernetes e2e suite [sig-node] Probing container should be restarted with a local redirect http liveness probe
Kubernetes e2e suite [sig-node] Probing container should be restarted with an exec liveness probe with timeout [MinimumKubeletVersion:1.20] [NodeConformance]
Kubernetes e2e suite [sig-node] Probing container should not be ready with an exec readiness probe timeout [MinimumKubeletVersion:1.20] [NodeConformance]
Kubernetes e2e suite [sig-node] Probing container should override timeoutGracePeriodSeconds when LivenessProbe field is set [Feature:ProbeTerminationGracePeriod]
Kubernetes e2e suite [sig-node] RuntimeClass should reject a Pod requesting a non-existent RuntimeClass [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] RuntimeClass should run a Pod requesting a RuntimeClass with scheduling with taints [Serial]
Kubernetes e2e suite [sig-node] RuntimeClass should schedule a Pod requesting a RuntimeClass and initialize its Overhead [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Secrets should be consumable from pods in env vars [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Security Context When creating a container with runAsNonRoot should not run with an explicit root user ID [LinuxOnly]
Kubernetes e2e suite [sig-node] Security Context should support pod.Spec.SecurityContext.RunAsUser [LinuxOnly]
Kubernetes e2e suite [sig-node] Security Context should support seccomp unconfined on the container [LinuxOnly]
Kubernetes e2e suite [sig-node] Security Context should support volume SELinux relabeling [Flaky] [LinuxOnly]
Kubernetes e2e suite [sig-node] Sysctls [LinuxOnly] [NodeConformance] should support sysctls [MinimumKubeletVersion:1.21] [Conformance]
Kubernetes e2e suite [sig-node] Variable Expansion should allow substituting values in a container's args [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Variable Expansion should fail substituting values in a volume subpath with backticks [Slow] [Conformance]
Kubernetes e2e suite [sig-node] Variable Expansion should verify that a failing subpath expansion can be modified during the lifecycle of a container [Slow] [Conformance]
Kubernetes e2e suite [sig-node] [Feature:Example] Downward API should create a pod that prints his name and namespace
Kubernetes e2e suite [sig-node] [Feature:Example] Secret should create a pod that reads a secret
Kubernetes e2e suite [sig-node] gpu Upgrade [Feature:GPUUpgrade] cluster upgrade should be able to run gpu pod after upgrade [Feature:GPUClusterUpgrade]
Kubernetes e2e suite [sig-node] gpu Upgrade [Feature:GPUUpgrade] master upgrade should NOT disrupt gpu pod [Feature:GPUMasterUpgrade]
Kubernetes e2e suite [sig-node] kubelet Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s.
Kubernetes e2e suite [sig-node] kubelet host cleanup with volume mounts [HostCleanup][Flaky] Host cleanup after disrupting NFS volume [NFS] after stopping the nfs-server and deleting the (sleeping) client pod, the NFS mount and the pod's UID directory should be removed.
Kubernetes e2e suite [sig-scheduling] SchedulerPredicates [Serial] PodTopologySpread Filtering validates 4 pods with MaxSkew=1 are evenly distributed into 2 nodes
Kubernetes e2e suite [sig-scheduling] SchedulerPredicates [Serial] validates local ephemeral storage resource limits of pods that are allowed to run [Feature:LocalStorageCapacityIsolation]
Kubernetes e2e suite [sig-scheduling] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance]
Kubernetes e2e suite [sig-scheduling] SchedulerPredicates [Serial] validates that taints-tolerations is respected if not matching
Kubernetes e2e suite [sig-scheduling] SchedulerPredicates [Serial] validates that there is no conflict between pods with same hostPort but different hostIP and protocol
Kubernetes e2e suite [sig-scheduling] SchedulerPreemption [Serial] PreemptionExecutionPath runs ReplicaSets to verify preemption running path [Conformance]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand Verify if offline PVC expansion works
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv written before kubelet restart is readable after restart.
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (block volmode)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup applied to the volume contents
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup skips ownership changes to the volume contents
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with mount options
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath directory is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath file is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should support creating multiple subpath from same volumes [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should support restarting containers using directory as subpath [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] subPath should unmount if pod is force deleted while kubelet is down [Disruptive][Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (delayed binding)] topology should fail to schedule a pod which has topologies that conflict with AllowedTopologies
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (delayed binding)] topology should provision a volume and schedule a pod with AllowedTopologies
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ext3)] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ext4)] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning should mount multiple PV pointing to the same storage on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning should provision storage with pvc data source
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning should provision storage with pvc data source in parallel [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should support existing directories when readOnly specified in the volumeSource
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should support existing directory
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should support existing single file [LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should support restarting containers using file as subpath [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Ephemeral Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should support two pods which have the same volume definition
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which have the same volume definition
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Inline-volume (default fs)] subPath should support creating multiple subpath from same volumes [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Inline-volume (default fs)] subPath should support existing single file [LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Inline-volume (default fs)] subPath should support readOnly directory specified in the volumeMount
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Inline-volume (default fs)] subPath should support restarting containers using directory as subpath [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Inline-volume (default fs)] subPath should unmount if pod is force deleted while kubelet is down [Disruptive][Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Inline-volume (default fs)] volumeIO should write files of various sizes, verify size, validate content [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv used in a pod that is deleted while the kubelet is down cleans up when the kubelet returns.
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv used in a pod that is force deleted while the kubelet is down cleans up when the kubelet returns.
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (block volmode)] volumeMode should fail to use a volume in a pod with mismatched mode [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (default fs)] subPath should fail if subpath directory is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (default fs)] subPath should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (default fs)] subPath should unmount if pod is force deleted while kubelet is down [Disruptive][Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (default fs)] subPath should unmount if pod is gracefully deleted while kubelet is down [Disruptive][Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (default fs)] volumeIO should write files of various sizes, verify size, validate content [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (ext3)] volumes should store data
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand Verify if offline PVC expansion works
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv used in a pod that is deleted while the kubelet is down cleans up when the kubelet returns.
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv written before kubelet restart is readable after restart.
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand Verify if offline PVC expansion works
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] capacity provides storage capacity information
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should support restarting containers using directory as subpath [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (default fs)] subPath should unmount if pod is gracefully deleted while kubelet is down [Disruptive][Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ext3)] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the single volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single volume from pods on different node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning should provision storage with any volume data source [Serial]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should verify container cannot write to subpath readonly volumes [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Dynamic PV (xfs)][Slow] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Ephemeral Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Ephemeral Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should create read-only inline ephemeral volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Generic Ephemeral-volume (default fs)] volumeLimits should support volume limits [Serial]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should fail if subpath file is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] subPath should support existing directories when readOnly specified in the volumeSource
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (default fs)] volumes should store data
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Inline-volume (xfs)][Slow] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] should concurrently access the single volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing directory
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support existing single file [LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should support readOnly directory specified in the volumeMount
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] subPath should unmount if pod is force deleted while kubelet is down [Disruptive][Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] volumeIO should write files of various sizes, verify size, validate content [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (default fs)] volumes should store data
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ext4)] volumes should store data
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single volume from pods on the same node
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode should fail to use a volume in a pod with mismatched mode [Slow]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (ntfs)][Feature:Windows] volumes should store data
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned PV (xfs)][Slow] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io][Serial] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works, check deletion (ephemeral)
Kubernetes e2e suite [sig-storage] CSI mock volume CSI FSGroupPolicy [LinuxOnly] should not modify fsGroup if fsGroupPolicy=None
Kubernetes e2e suite [sig-storage] CSI mock volume CSI Volume expansion should expand volume by restarting pod if attach=off, nodeExpansion=on
Kubernetes e2e suite [sig-storage] CSI mock volume CSI online volume expansion should expand volume without restarting pod if attach=off, nodeExpansion=on
Kubernetes e2e suite [sig-storage] CSI mock volume CSI volume limit information using mock driver should report attach limit for generic ephemeral volume when persistent volume is attached [Slow]
Kubernetes e2e suite [sig-storage] CSI mock volume CSI volume limit information using mock driver should report attach limit when limit is bigger than 0 [Slow]
Kubernetes e2e suite [sig-storage] CSI mock volume CSIServiceAccountToken token should be plumbed down when csiServiceAccountTokenEnabled=true
Kubernetes e2e suite [sig-storage] CSI mock volume CSIServiceAccountToken token should not be plumbed down when csiServiceAccountTokenEnabled=false
Kubernetes e2e suite [sig-storage] CSI mock volume CSIStorageCapacity CSIStorageCapacity unused
Kubernetes e2e suite [sig-storage] CSI mock volume Delegate FSGroup to CSI driver [LinuxOnly] should pass FSGroup to CSI driver if it is set in pod and driver supports VOLUME_MOUNT_GROUP
Kubernetes e2e suite [sig-storage] CSI mock volume storage capacity unlimited
Kubernetes e2e suite [sig-storage] ConfigMap updates should be reflected in volume [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] Downward API [Serial] [Disruptive] [Feature:EphemeralStorage] Downward API tests for local ephemeral storage should provide default limits.ephemeral-storage from node allocatable
Kubernetes e2e suite [sig-storage] Downward API volume should provide container's cpu limit [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] Downward API volume should provide node allocatable (cpu) as default cpu limit if the limit is not set [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] Downward API volume should provide node allocatable (memory) as default memory limit if the limit is not set [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] Downward API volume should update annotations on modification [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] Dynamic Provisioning DynamicProvisioner [Slow] [Feature:StorageProvider] should provision storage with non-default reclaim policy Retain
Kubernetes e2e suite [sig-storage] EmptyDir volumes should support (root,0644,default) [LinuxOnly] [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] EmptyDir volumes should support (root,0777,default) [LinuxOnly] [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] EmptyDir volumes should support (root,0777,tmpfs) [LinuxOnly] [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] EmptyDir volumes volume on default medium should have the correct mode [LinuxOnly] [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] new files should be created with FSGroup ownership when container is root
Kubernetes e2e suite [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] volume on default medium should have the correct mode using FSGroup
Kubernetes e2e suite [sig-storage] EmptyDir wrapper volumes should not cause race condition when used for configmaps [Serial] [Conformance]
Kubernetes e2e suite [sig-storage] Ephemeralstorage When pod refers to non-existent ephemeral storage should allow deletion of pod with invalid volume : configmap
Kubernetes e2e suite [sig-storage] Ephemeralstorage When pod refers to non-existent ephemeral storage should allow deletion of pod with invalid volume : secret
Kubernetes e2e suite [sig-storage] Flexvolumes should be mountable when attachable [Feature:Flexvolumes]
Kubernetes e2e suite [sig-storage] GenericPersistentVolume[Disruptive] When kubelet restarts Should test that a file written to the mount before kubelet restart is readable after restart.
Kubernetes e2e suite [sig-storage] GenericPersistentVolume[Disruptive] When kubelet restarts Should test that a volume mounted to a pod that is force deleted while the kubelet is down unmounts when the kubelet returns.
Kubernetes e2e suite [sig-storage] HostPath should support r/w [NodeConformance]
Kubernetes e2e suite [sig-storage] HostPath should support subPath [NodeConformance]
Kubernetes e2e suite [sig-storage] HostPathType Block Device [Slow] Should fail on mounting block device 'ablkdev' when HostPathType is HostPathSocket
Kubernetes e2e suite [sig-storage] HostPathType Character Device [Slow] Should fail on mounting character device 'achardev' when HostPathType is HostPathBlockDev
Kubernetes e2e suite [sig-storage] HostPathType Character Device [Slow] Should fail on mounting character device 'achardev' when HostPathType is HostPathDirectory
Kubernetes e2e suite [sig-storage] HostPathType Directory [Slow] Should fail on mounting non-existent directory 'does-not-exist-dir' when HostPathType is HostPathDirectory
Kubernetes e2e suite [sig-storage] HostPathType File [Slow] Should be able to mount file 'afile' successfully when HostPathType is HostPathFile
Kubernetes e2e suite [sig-storage] HostPathType Socket [Slow] Should be able to mount socket 'asocket' successfully when HostPathType is HostPathUnset
Kubernetes e2e suite [sig-storage] HostPathType Socket [Slow] Should fail on mounting socket 'asocket' when HostPathType is HostPathCharDev
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand Verify if offline PVC expansion works
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv used in a pod that is deleted while the kubelet is down cleans up when the kubelet returns.
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv used in a pod that is force deleted while the kubelet is down cleans up when the kubelet returns.
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the single volume from pods on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with any volume data source [Serial]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with mount options
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with pvc data source in parallel [Slow]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand should resize volume when PVC is edited while pod is using it
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup applied to the volume contents
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)] provisioning should mount multiple PV pointing to the same storage on the same node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath directory is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath file is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)] subPath should support restarting containers using file as subpath [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (delayed binding)] topology should fail to schedule a pod which has topologies that conflict with AllowedTopologies
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext3)] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext3)] volumes should store data
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the single volume from pods on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the single volume from pods on the same node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single volume from pods on the same node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should fail to use a volume in a pod with mismatched mode [Slow]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)(allowExpansion)][Feature:Windows] volume-expand Verify if offline PVC expansion works
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)(allowExpansion)][Feature:Windows] volume-expand should resize volume when PVC is edited while pod is using it
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should support readOnly directory specified in the volumeMount
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should support restarting containers using directory as subpath [Slow]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath should verify container cannot write to subpath readonly volumes [Slow]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] volumeIO should write files of various sizes, verify size, validate content [Slow]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] volumes should store data
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should concurrently access the single volume from pods on different node
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (xfs)][Slow] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (xfs)][Slow] volumes should store data
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (default fs)] subPath should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (default fs)] subPath should support creating multiple subpath from same volumes [Slow]
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (default fs)] subPath should support existing directories when readOnly specified in the volumeSource
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (ext4)] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (ntfs)][Feature:Windows] volumes should allow exec of files on the volume
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Inline-volume (xfs)][Slow] volumes should store data
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Pre-provisioned PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv used in a pod that is deleted while the kubelet is down cleans up when the kubelet returns.
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Pre-provisioned PV (block volmode)] disruptive[Disruptive][LinuxOnly] Should test that pv used in a pod that is force deleted while the kubelet is down cleans up when the kubelet returns.
Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node