ci-kubernetes-e2e-snapshot #1529335867091456000
Recent runs || View in Spyglass
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\sAfterSuite$'
test/e2e/e2e.go:87
May 25 07:38:11.961: The framework ClientSet must not be nil at this point
test/e2e/framework/cleanup.go:76
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\]\sprovisioning\sshould\sprovision\sstorage\swith\ssnapshot\sdata\ssource\s\[Feature\:VolumeSnapshotDataSource\]$'
test/e2e/storage/testsuites/provisioning.go:208
May 25 06:24:58.052: Unexpected error:
<*errors.errorString | 0xc0030ca450>: {
s: "VolumeSnapshot snapshot-gwgj2 is not ready within 5m0s",
}
VolumeSnapshot snapshot-gwgj2 is not ready within 5m0s
occurred
test/e2e/storage/utils/snapshot.go:86
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
test/e2e/framework/framework.go:186
�[1mSTEP�[0m: Creating a kubernetes client
May 25 06:19:30.520: INFO: >>> kubeConfig: /workspace/.kube/config
�[1mSTEP�[0m: Building a namespace api object, basename provisioning
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
�[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace
[It] should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
test/e2e/storage/testsuites/provisioning.go:208
�[1mSTEP�[0m: deploying csi gce-pd driver
�[1mSTEP�[0m: Building a driver namespace object, basename provisioning-8483
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
May 25 06:19:31.139: INFO: Found CI service account key at /etc/service-account/service-account.json
May 25 06:19:31.139: INFO: Running cp [/etc/service-account/service-account.json /tmp/c68c6f8a-7776-4027-ba71-02f478c96b61/cloud-sa.json]
May 25 06:19:31.182: INFO: Shredding file /tmp/c68c6f8a-7776-4027-ba71-02f478c96b61/cloud-sa.json
May 25 06:19:31.182: INFO: Running shred [--remove /tmp/c68c6f8a-7776-4027-ba71-02f478c96b61/cloud-sa.json]
May 25 06:19:31.216: INFO: File /tmp/c68c6f8a-7776-4027-ba71-02f478c96b61/cloud-sa.json successfully shredded
May 25 06:19:31.227: INFO: creating *v1.ServiceAccount: provisioning-8483-569/csi-attacher
May 25 06:19:31.271: INFO: creating *v1.ClusterRole: external-attacher-runner-provisioning-8483
May 25 06:19:31.271: INFO: Define cluster role external-attacher-runner-provisioning-8483
May 25 06:19:31.312: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-provisioning-8483
May 25 06:19:31.354: INFO: creating *v1.Role: provisioning-8483-569/external-attacher-cfg-provisioning-8483
May 25 06:19:31.395: INFO: creating *v1.RoleBinding: provisioning-8483-569/csi-attacher-role-cfg
May 25 06:19:31.436: INFO: creating *v1.ServiceAccount: provisioning-8483-569/csi-provisioner
May 25 06:19:31.476: INFO: creating *v1.ClusterRole: external-provisioner-runner-provisioning-8483
May 25 06:19:31.476: INFO: Define cluster role external-provisioner-runner-provisioning-8483
May 25 06:19:31.519: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-8483
May 25 06:19:31.560: INFO: creating *v1.Role: provisioning-8483-569/external-provisioner-cfg-provisioning-8483
May 25 06:19:31.601: INFO: creating *v1.RoleBinding: provisioning-8483-569/csi-provisioner-role-cfg
May 25 06:19:31.642: INFO: creating *v1.ServiceAccount: provisioning-8483-569/csi-gce-pd-controller-sa
May 25 06:19:31.683: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-provisioning-8483
May 25 06:19:31.683: INFO: Define cluster role csi-gce-pd-provisioner-role-provisioning-8483
May 25 06:19:31.725: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-provisioning-8483
May 25 06:19:31.766: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-provisioning-8483
May 25 06:19:31.766: INFO: Define cluster role csi-gce-pd-attacher-role-provisioning-8483
May 25 06:19:31.806: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-provisioning-8483
May 25 06:19:31.848: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-provisioning-8483
May 25 06:19:31.848: INFO: Define cluster role csi-gce-pd-resizer-role-provisioning-8483
May 25 06:19:31.890: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-provisioning-8483
May 25 06:19:31.932: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-provisioning-8483
May 25 06:19:31.932: INFO: Define cluster role csi-gce-pd-snapshotter-role-provisioning-8483
May 25 06:19:31.974: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-provisioning-8483
May 25 06:19:32.016: INFO: creating *v1.Role: provisioning-8483-569/csi-gce-pd-leaderelection-role-provisioning-8483
May 25 06:19:32.057: INFO: creating *v1.RoleBinding: provisioning-8483-569/csi-gce-pd-controller-leaderelection-binding
May 25 06:19:32.099: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-provisioning-8483
May 25 06:19:32.139: INFO: creating *v1.DaemonSet: provisioning-8483-569/csi-gce-pd-node
May 25 06:19:32.184: INFO: creating *v1.StatefulSet: provisioning-8483-569/csi-gce-pd-controller
May 25 06:19:32.356: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-gw73
May 25 06:19:37.536: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-kn6c
May 25 06:19:37.578: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-nndt
May 25 06:19:37.618: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
May 25 06:19:37.618: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
May 25 06:19:37.618: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-84832jkgr 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Provisioner:pd.csi.storage.gke.io,Parameters:map[string]string{type: pd-standard,},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-8483 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-84832jkgr,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-8483 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-84832jkgr,VolumeMode:nil,DataSource:nil,DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}
�[1mSTEP�[0m: Creating a StorageClass
�[1mSTEP�[0m: [Initialize dataSource]creating a initClaim
�[1mSTEP�[0m: starting gcepd-injector
May 25 06:19:37.832: INFO: Waiting up to 5m0s for pod "gcepd-injector" in namespace "provisioning-8483" to be "running"
May 25 06:19:37.875: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 43.209291ms
May 25 06:19:39.917: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 2.085568622s
May 25 06:19:41.916: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 4.083966225s
May 25 06:19:43.917: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 6.084869568s
May 25 06:19:45.916: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 8.084017185s
May 25 06:19:47.916: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 10.084348799s
May 25 06:19:49.917: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 12.085040496s
May 25 06:19:51.917: INFO: Pod "gcepd-injector": Phase="Pending", Reason="", readiness=false. Elapsed: 14.084868843s
May 25 06:19:53.921: INFO: Pod "gcepd-injector": Phase="Running", Reason="", readiness=true. Elapsed: 16.089082217s
May 25 06:19:53.921: INFO: Pod "gcepd-injector" satisfied condition "running"
�[1mSTEP�[0m: Writing text file contents in the container.
May 25 06:19:53.921: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.82.64.164 --kubeconfig=/workspace/.kube/config --namespace=provisioning-8483 exec gcepd-injector --namespace=provisioning-8483 -- /bin/sh -c echo 'Hello from namespace provisioning-8483' > /opt/0/index.html'
May 25 06:19:54.407: INFO: stderr: ""
May 25 06:19:54.407: INFO: stdout: ""
�[1mSTEP�[0m: Checking that text file contents are perfect.
May 25 06:19:54.407: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.82.64.164 --kubeconfig=/workspace/.kube/config --namespace=provisioning-8483 exec gcepd-injector --namespace=provisioning-8483 -- cat /opt/0/index.html'
May 25 06:19:54.875: INFO: stderr: ""
May 25 06:19:54.875: INFO: stdout: "Hello from namespace provisioning-8483\n"
May 25 06:19:54.875: INFO: ExecWithOptions {Command:[/bin/sh -c test -d /opt/0] Namespace:provisioning-8483 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
May 25 06:19:54.875: INFO: >>> kubeConfig: /workspace/.kube/config
May 25 06:19:54.877: INFO: ExecWithOptions: Clientset creation
May 25 06:19:54.877: INFO: ExecWithOptions: execute(POST https://34.82.64.164/api/v1/namespaces/provisioning-8483/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-d+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true)
May 25 06:19:55.199: INFO: ExecWithOptions {Command:[/bin/sh -c test -b /opt/0] Namespace:provisioning-8483 PodName:gcepd-injector ContainerName:gcepd-injector Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
May 25 06:19:55.199: INFO: >>> kubeConfig: /workspace/.kube/config
May 25 06:19:55.201: INFO: ExecWithOptions: Clientset creation
May 25 06:19:55.201: INFO: ExecWithOptions: execute(POST https://34.82.64.164/api/v1/namespaces/provisioning-8483/pods/gcepd-injector/exec?command=%2Fbin%2Fsh&command=-c&command=test+-b+%2Fopt%2F0&container=gcepd-injector&container=gcepd-injector&stderr=true&stdout=true)
�[1mSTEP�[0m: Deleting pod gcepd-injector in namespace provisioning-8483
May 25 06:19:55.558: INFO: Waiting for pod gcepd-injector to disappear
May 25 06:19:55.598: INFO: Pod gcepd-injector still exists
May 25 06:19:57.599: INFO: Waiting for pod gcepd-injector to disappear
May 25 06:19:57.640: INFO: Pod gcepd-injector no longer exists
�[1mSTEP�[0m: creating a SnapshotClass
�[1mSTEP�[0m: creating a dynamic VolumeSnapshot
May 25 06:19:57.774: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-gwgj2 to become ready
May 25 06:19:57.819: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:19:59.865: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:01.906: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:03.948: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:05.990: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:08.032: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:10.075: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:12.116: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:14.157: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:16.198: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:18.241: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:20.282: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:22.324: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:24.368: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:26.410: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:28.452: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:30.494: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:32.538: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:34.580: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:36.622: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:38.665: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:40.707: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:42.750: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:44.791: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:46.833: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:48.875: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:50.917: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:52.960: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:55.009: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:57.051: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:20:59.093: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:01.134: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:03.176: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:05.221: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:07.262: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:09.303: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:11.379: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:13.422: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:15.464: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:17.506: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:19.547: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:21.589: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:23.631: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:25.672: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:27.714: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:29.756: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:31.799: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:33.841: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:35.883: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:37.925: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:39.967: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:42.009: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:44.050: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:46.093: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:48.134: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:50.177: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:52.219: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:54.264: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:56.305: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:21:58.354: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:00.396: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:02.437: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:04.479: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:06.521: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:08.563: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:10.604: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:12.647: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:14.689: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:16.731: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:18.773: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:20.816: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:22.870: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:24.920: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:26.962: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:29.002: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:31.044: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:33.085: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:35.128: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:37.171: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:39.213: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:41.254: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:43.295: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:45.337: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:47.379: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:49.420: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:51.462: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:53.504: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:55.547: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:57.589: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:22:59.632: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:01.674: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:03.715: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:05.758: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:07.804: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:09.847: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:11.893: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:13.935: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:15.976: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:18.017: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:20.060: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:22.103: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:24.144: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:26.187: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:28.230: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:30.271: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:32.313: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:34.355: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:36.397: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:38.439: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:40.482: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:42.524: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:44.566: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:46.609: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:48.652: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:50.694: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:52.736: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:54.777: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:56.819: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:23:58.864: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:00.905: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:02.947: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:04.990: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:07.032: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:09.073: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:11.114: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:13.156: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:15.199: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:17.241: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:19.283: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:21.327: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:23.368: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:25.410: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:27.451: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:29.493: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:31.535: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:33.577: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:35.619: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:37.661: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:39.703: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:41.744: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:43.787: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:45.830: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:47.871: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:49.913: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:51.954: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:53.997: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:56.051: INFO: VolumeSnapshot snapshot-gwgj2 found but is not ready.
May 25 06:24:58.051: INFO: WaitUntil failed after reaching the timeout 5m0s
May 25 06:24:58.052: FAIL: Unexpected error:
<*errors.errorString | 0xc0030ca450>: {
s: "VolumeSnapshot snapshot-gwgj2 is not ready within 5m0s",
}
VolumeSnapshot snapshot-gwgj2 is not ready within 5m0s
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eab80, 0xc0026872c0?}, 0xc002bca938)
test/e2e/storage/utils/snapshot.go:86 +0x1ad
k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f220008b9b0, 0xc000569ee0}, 0xc0039c1bc0, {{0x7179f31, 0x17}, {0x0, 0x0}, {0x7133ab2, 0x9}, {0x0, ...}, ...}, ...)
test/e2e/storage/framework/snapshot_resource.go:92 +0x246
k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning(_, {{0xc0034fc288, 0x11}, {0x712a523, 0x5}, {0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...)
test/e2e/storage/testsuites/provisioning.go:1083 +0x41f
k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4()
test/e2e/storage/testsuites/provisioning.go:227 +0x485
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
�[1mSTEP�[0m: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-8483 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-84832jkgr,VolumeMode:nil,DataSource:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-gwgj2,},DataSourceRef:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}
�[1mSTEP�[0m: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-84832jkgr 5f9e208a-0b58-482b-b83e-a51ac8a43cb3 7204 0 2022-05-25 06:19:37 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update storage.k8s.io/v1 2022-05-25 06:19:37 +0000 UTC FieldsV1 {"f:parameters":{".":{},"f:type":{}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}} }]},Provisioner:pd.csi.storage.gke.io,Parameters:map[string]string{type: pd-standard,},ReclaimPolicy:*Delete,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-9hpm7 pvc- provisioning-8483 99519066-e437-4e4d-a148-2c2fdcd70d0f 8135 0 2022-05-25 06:24:58 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection] [{e2e.test Update v1 2022-05-25 06:24:58 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:dataSource":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}} }]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{5368709120 0} {<nil>} 5Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-84832jkgr,VolumeMode:*Filesystem,DataSource:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-gwgj2,},DataSourceRef:&TypedLocalObjectReference{APIGroup:*snapshot.storage.k8s.io,Kind:VolumeSnapshot,Name:snapshot-gwgj2,},},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},AllocatedResources:ResourceList{},ResizeStatus:nil,},}
May 25 06:24:58.183: INFO: Waiting up to 5m0s for pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f" in namespace "provisioning-8483" to be "running"
May 25 06:24:58.223: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 40.223451ms
May 25 06:25:00.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2.082333317s
May 25 06:25:02.305: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4.12207557s
May 25 06:25:04.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 6.082523993s
May 25 06:25:06.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 8.082365733s
May 25 06:25:08.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 10.081627287s
May 25 06:25:10.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 12.083438423s
May 25 06:25:12.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 14.08226688s
May 25 06:25:14.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 16.08168691s
May 25 06:25:16.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 18.082467825s
May 25 06:25:18.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 20.081834511s
May 25 06:25:20.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 22.081951437s
May 25 06:25:22.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 24.082147877s
May 25 06:25:24.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 26.08192193s
May 25 06:25:26.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 28.082665226s
May 25 06:25:28.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 30.082433105s
May 25 06:25:30.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 32.081771659s
May 25 06:25:32.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 34.081761019s
May 25 06:25:34.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 36.082104768s
May 25 06:25:36.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 38.08324629s
May 25 06:25:38.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 40.082106735s
May 25 06:25:40.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 42.082198892s
May 25 06:25:42.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 44.083256001s
May 25 06:25:44.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 46.081850937s
May 25 06:25:46.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 48.081732692s
May 25 06:25:48.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 50.081691739s
May 25 06:25:50.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 52.081723184s
May 25 06:25:52.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 54.082210752s
May 25 06:25:54.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 56.082976518s
May 25 06:25:56.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 58.082781016s
May 25 06:25:58.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.082066587s
May 25 06:26:00.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.08186331s
May 25 06:26:02.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.082056723s
May 25 06:26:04.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.082458974s
May 25 06:26:06.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.081417377s
May 25 06:26:08.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.081710857s
May 25 06:26:10.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.081864151s
May 25 06:26:12.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.081774126s
May 25 06:26:14.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.081410428s
May 25 06:26:16.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.081903485s
May 25 06:26:18.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.082735112s
May 25 06:26:20.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.082204346s
May 25 06:26:22.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.082529207s
May 25 06:26:24.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.082427891s
May 25 06:26:26.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.081436192s
May 25 06:26:28.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.082134905s
May 25 06:26:30.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.081996095s
May 25 06:26:32.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.082481887s
May 25 06:26:34.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.082035646s
May 25 06:26:36.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.081934758s
May 25 06:26:38.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.082355287s
May 25 06:26:40.271: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.088270342s
May 25 06:26:42.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.082577925s
May 25 06:26:44.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.083525956s
May 25 06:26:46.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.081373094s
May 25 06:26:48.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.081106737s
May 25 06:26:50.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.082526893s
May 25 06:26:52.282: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.098976654s
May 25 06:26:54.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.081709725s
May 25 06:26:56.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.080764096s
May 25 06:26:58.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.081975438s
May 25 06:27:00.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.082246938s
May 25 06:27:02.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.082452213s
May 25 06:27:04.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.081806705s
May 25 06:27:06.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.081629955s
May 25 06:27:08.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.08182443s
May 25 06:27:10.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.082259722s
May 25 06:27:12.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.082196033s
May 25 06:27:14.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.081835316s
May 25 06:27:16.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.08169254s
May 25 06:27:18.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.082234311s
May 25 06:27:20.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.081673655s
May 25 06:27:22.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.082547683s
May 25 06:27:24.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.082337804s
May 25 06:27:26.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.081037612s
May 25 06:27:28.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.081917559s
May 25 06:27:30.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.082314718s
May 25 06:27:32.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.082647727s
May 25 06:27:34.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.082783653s
May 25 06:27:36.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.081907972s
May 25 06:27:38.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.083741667s
May 25 06:27:40.269: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.085916031s
May 25 06:27:42.269: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.085958478s
May 25 06:27:44.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.081818802s
May 25 06:27:46.269: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.085797435s
May 25 06:27:48.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.081148687s
May 25 06:27:50.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.082138598s
May 25 06:27:52.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.08209156s
May 25 06:27:54.267: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.0838141s
May 25 06:27:56.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.081939688s
May 25 06:27:58.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.081345089s
May 25 06:28:00.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.08245705s
May 25 06:28:02.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.08136607s
May 25 06:28:04.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.081903575s
May 25 06:28:06.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.082974931s
May 25 06:28:08.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.081885643s
May 25 06:28:10.276: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.093012727s
May 25 06:28:12.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.082838786s
May 25 06:28:14.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.081922824s
May 25 06:28:16.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.082783347s
May 25 06:28:18.263: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.080659077s
May 25 06:28:20.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.083172359s
May 25 06:28:22.266: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.08290219s
May 25 06:28:24.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.082191712s
May 25 06:28:26.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.082315613s
May 25 06:28:28.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.081802437s
May 25 06:28:30.284: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.101368405s
May 25 06:28:32.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.08198674s
May 25 06:28:34.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.080855488s
May 25 06:28:36.264: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.081491758s
May 25 06:28:38.293: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.1104483s
May 25 06:28:40.267: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.083850385s
May 25 06:28:42.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.082267341s
May 25 06:28:44.265: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.082711206s
May 25 06:28:46.278: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.094832653s
May 25 06:28:48.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.091500085s
May 25 06:28:50.278: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.095365269s
May 25 06:28:52.276: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.09310399s
May 25 06:28:54.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.091597016s
May 25 06:28:56.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.091700315s
May 25 06:28:58.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.091163978s
May 25 06:29:00.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.091358931s
May 25 06:29:02.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.091429792s
May 25 06:29:04.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.091241597s
May 25 06:29:06.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.09126113s
May 25 06:29:08.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.091646602s
May 25 06:29:10.276: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.092808509s
May 25 06:29:12.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.091073884s
May 25 06:29:14.275: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.091900897s
May 25 06:29:16.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.091514992s
May 25 06:29:18.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.091090524s
May 25 06:29:20.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.091490257s
May 25 06:29:22.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.091579874s
May 25 06:29:24.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.09110447s
May 25 06:29:26.275: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.091897575s
May 25 06:29:28.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.091445948s
May 25 06:29:30.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.0915244s
May 25 06:29:32.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.091588472s
May 25 06:29:34.275: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.09230227s
May 25 06:29:36.273: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.090577791s
May 25 06:29:38.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.09123321s
May 25 06:29:40.275: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.091890468s
May 25 06:29:42.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.091674804s
May 25 06:29:44.275: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.091814918s
May 25 06:29:46.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.091737799s
May 25 06:29:48.275: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.091945576s
May 25 06:29:50.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.091476033s
May 25 06:29:52.268: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.085718527s
May 25 06:29:54.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.091669703s
May 25 06:29:56.274: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.091106144s
May 25 06:29:58.275: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.091906459s
May 25 06:29:58.375: INFO: Pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.191948178s
May 25 06:29:58.376: INFO: Timed out while waiting for pod provisioning-8483/pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f to be running. Last observed as: <*v1.Pod | 0xc0036a3400>: {
TypeMeta: {Kind: "", APIVersion: ""},
ObjectMeta: {
Name: "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f",
GenerateName: "",
Namespace: "provisioning-8483",
SelfLink: "",
UID: "76dcec6c-68f2-4cba-a129-c4da33ac0928",
ResourceVersion: "8136",
Generation: 0,
CreationTimestamp: {
Time: 2022-05-25T06:24:58Z,
},
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: nil,
Annotations: nil,
OwnerReferences: nil,
Finalizers: nil,
ManagedFields: [
{
Manager: "e2e.test",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: 2022-05-25T06:24:58Z,
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"write-pod\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:securityContext\":{\".\":{},\"f:privileged\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/mnt/volume1\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{\".\":{},\"f:fsGroup\":{}},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"volume1\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}",
},
Subresource: "",
},
],
},
Spec: {
Volumes: [
{
Name: "volume1",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: {
ClaimName: "pvc-9hpm7",
ReadOnly: false,
},
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: nil,
PortworxVolume: nil,
ScaleIO: nil,
StorageOS: nil,
CSI: nil,
Ephemeral: nil,
},
},
{
Name: "kube-api-access-rg7kv",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: {
Sources: [
{
Secret: nil,
DownwardAPI: nil,
ConfigMap: nil,
ServiceAccountToken: {
Audience: "",
ExpirationSeconds: 3607,
Path: "token",
},
},
{
Secret: nil,
DownwardAPI: nil,
ConfigMap: {
LocalObjectReference: {Name: ...},
Items: [...],
Optional: nil,
},
ServiceAccountToken: nil,
},
{
Secret: nil,
DownwardAPI: {Items: [...]},
ConfigMap: nil,
ServiceAccountToken: nil,
},
],
DefaultMode: 420,
},
PortworxVolume: nil,
ScaleIO: nil,
StorageOS: nil,
CSI: nil,
Ephemeral: nil,
},
},
],
InitContainers: nil,
Containers: [
{
Name: "write-pod",
Image: "k8s.gcr.io/e2e-test-images/busybox:1.29-2",
Command: [
"/bin/sh",
"-c",
"trap exit TERM; while true; do sleep 1; done",
],
Args: nil,
WorkingDir: "",
Ports: nil,
EnvFrom: nil,
Env: nil,
Resources: {Limits: nil, Requests: nil},
VolumeMounts: [
{
Name: "volume1",
ReadOnly: false,
MountPath: "/mnt/volume1",
SubPath: "",
MountPropagation: nil,
SubPathExpr: "",
},
{
Name: "kube-api-access-rg7kv",
ReadOnly: true,
MountPath: "/var/run/secrets/kubernetes.io/serviceaccount",
SubPath: "",
MountPropagation: nil,
SubPathExpr: "",
},
],
VolumeDevices: nil,
LivenessProbe: nil,
ReadinessProbe: nil,
StartupProbe: nil,
Lifecycle: nil,
TerminationMessagePath: "/dev/termination-log",
TerminationMessagePolicy: "File",
ImagePullPolicy: "IfNotPresent",
SecurityContext: {
Capabilities: nil,
Privileged: false,
SELinuxOptions: nil,
WindowsOptions: nil,
RunAsUser: nil,
RunAsGroup: nil,
RunAsNonRoot: nil,
ReadOnlyRootFilesystem: nil,
AllowPrivilegeEscalation: nil,
ProcMount: nil,
SeccompProfile: nil,
},
Stdin: false,
StdinOnce: false,
TTY: false,
},
],
EphemeralContainers: nil,
RestartPolicy: "OnFailure",
TerminationGracePeriodSeconds: 30,
ActiveDeadlineSeconds: nil,
DNSPolicy: "ClusterFirst",
NodeSelector: nil,
ServiceAccountName: "default",
DeprecatedServiceAccount: "default",
AutomountServiceAccountToken: nil,
NodeName: "",
HostNetwork: false,
HostPID: false,
HostIPC: false,
ShareProcessNamespace: nil,
SecurityContext: {
SELinuxOptions: nil,
WindowsOptions: nil,
RunAsUser: nil,
RunAsGroup: nil,
RunAsNonRoot: nil,
SupplementalGroups: nil,
FSGroup: 1000,
Sysctls: nil,
FSGroupChangePolicy: nil,
SeccompProfile: nil,
},
ImagePullSecrets: nil,
Hostname: "",
Subdomain: "",
Affinity: nil,
SchedulerName: "default-scheduler",
Tolerations: [
{
Key: "node.kubernetes.io/not-ready",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: 300,
},
{
Key: "node.kubernetes.io/unreachable",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: 300,
},
],
HostAliases: nil,
PriorityClassName: "",
Priority: 0,
DNSConfig: nil,
ReadinessGates: nil,
RuntimeClassName: nil,
EnableServiceLinks: true,
PreemptionPolicy: "PreemptLowerPriority",
Overhead: nil,
TopologySpreadConstraints: nil,
SetHostnameAsFQDN: nil,
OS: nil,
},
Status: {
Phase: "Pending",
Conditions: nil,
Message: "",
Reason: "",
NominatedNodeName: "",
HostIP: "",
PodIP: "",
PodIPs: nil,
StartTime: nil,
InitContainerStatuses: nil,
ContainerStatuses: nil,
QOSClass: "BestEffort",
EphemeralContainerStatuses: nil,
},
}
May 25 06:29:58.379: FAIL: Unexpected error:
<*errors.errorString | 0xc00066b7a0>: {
s: "pod \"pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f\" is not Running: timed out while waiting for pod provisioning-8483/pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f to be running",
}
pod "pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f" is not Running: timed out while waiting for pod provisioning-8483/pod-1f264e90-0d46-4e82-8323-7bd673d4cd0f to be running
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning({{0x7a45288, 0xc002a4a600}, 0xc0012cd980, 0xc0008356c0, 0xc000835880, 0xc002148480, {0x0, 0x0}, {0x0, 0x0, ...}, ...})
test/e2e/storage/testsuites/provisioning.go:624 +0x808
k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4()
test/e2e/storage/testsuites/provisioning.go:243 +0x618
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
May 25 06:29:58.380: INFO: deleting claim "provisioning-8483"/"pvc-9hpm7"
May 25 06:29:58.476: INFO: deleting initClaim "provisioning-8483"/"pvc-bjfzl"
May 25 06:29:58.578: INFO: deleting snapshot "provisioning-8483"/"snapshot-gwgj2"
May 25 06:29:58.619: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c readyToUse:false]
May 25 06:29:58.619: INFO: snapshotContentName snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c
May 25 06:29:58.705: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c to be deleted
May 25 06:29:58.754: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:00.795: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:02.838: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:04.879: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:06.920: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:08.962: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:11.011: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:13.054: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:15.102: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:17.144: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:19.185: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:21.226: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:23.269: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:25.311: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:27.352: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:29.394: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:31.435: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:33.477: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:35.519: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:37.561: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:39.602: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:41.644: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:43.686: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:45.728: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:47.771: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:49.813: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:51.855: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:53.902: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:55.944: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:30:57.989: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:00.031: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:02.073: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:04.116: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:06.158: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:08.200: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:10.242: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:12.284: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:14.330: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:16.375: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:18.417: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:20.460: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:22.502: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:24.544: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:26.585: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:28.628: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:30.670: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:32.713: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:34.754: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:36.797: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:38.839: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:40.880: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:42.922: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:44.964: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:47.006: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:49.049: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:51.092: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:53.134: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:55.179: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:57.220: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:31:59.262: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:01.305: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:03.347: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:05.388: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:07.430: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:09.471: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:11.512: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:13.554: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:15.597: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:17.639: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:19.760: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:21.801: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:23.843: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:25.884: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:27.927: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:29.969: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:32.012: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:34.054: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:36.095: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:38.138: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:40.181: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:42.223: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:44.288: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:46.329: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:48.371: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:50.414: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:52.469: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:54.511: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:56.555: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:32:58.603: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:00.646: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:02.688: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:04.731: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:06.773: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:08.815: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:10.859: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:12.903: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:14.945: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:16.986: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:19.027: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:21.071: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:23.113: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:25.155: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:27.198: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:29.241: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:31.282: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:33.329: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:35.371: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:37.413: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:39.455: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:41.497: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:43.540: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:45.582: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:47.627: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:49.668: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:51.711: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:53.754: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:55.798: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:57.841: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:33:59.882: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:01.925: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:03.968: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:06.011: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:08.054: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:10.097: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:12.139: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:14.181: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:16.227: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:18.269: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:20.312: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:22.356: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:24.399: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:26.448: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:28.496: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:30.539: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:32.580: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:34.622: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:36.665: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:38.708: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:40.753: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:42.796: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:44.846: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:46.889: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:48.932: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:50.974: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:53.017: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:55.060: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:57.103: INFO: volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c has been found and is not deleted
May 25 06:34:59.104: INFO: WaitUntil failed after reaching the timeout 5m0s
May 25 06:34:59.105: FAIL: Unexpected error:
<*errors.errorString | 0xc0018bce00>: {
s: "volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c is not deleted within 5m0s",
}
volumesnapshotcontents snapcontent-966b81a2-5206-42f4-aa5d-a54ba2af081c is not deleted within 5m0s
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/framework.(*SnapshotResource).CleanupResource(0xc000bd5130, 0xc0012cd980)
test/e2e/storage/framework/snapshot_resource.go:204 +0xefd
k8s.io/kubernetes/test/e2e/storage/testsuites.prepareSnapshotDataSourceForProvisioning.func1()
test/e2e/storage/testsuites/provisioning.go:1098 +0x294
panic({0x6c82620, 0xc003331280})
/usr/local/go/src/runtime/panic.go:838 +0x207
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1()
test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d
panic({0x6309f20, 0x79de560})
/usr/local/go/src/runtime/panic.go:838 +0x207
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc0002f5680, 0x1c5}, {0xc002e2c888?, 0x7126d2b?, 0xc002e2c8a8?})
test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197
k8s.io/kubernetes/test/e2e/framework.Fail({0xc003900380, 0x1b0}, {0xc0037c5c20?, 0xc003900380?, 0xc0031ee9c0?})
test/e2e/framework/log.go:63 +0x145
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc002e2c9f0, {0x7a027f8, 0xa670820}, 0x0, {0x0, 0x0, 0x0})
vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc002e2c9f0, {0x7a027f8, 0xa670820}, {0x0, 0x0, 0x0})
vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92
k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x7a45288?, {0x79e2180?, 0xc00066b7a0?}, {0x0?, 0x2?, 0x0?})
test/e2e/framework/expect.go:46 +0x9d
k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...)
test/e2e/framework/expect.go:40
k8s.io/kubernetes/test/e2e/storage/testsuites.StorageClassTest.TestDynamicProvisioning({{0x7a45288, 0xc002a4a600}, 0xc0012cd980, 0xc0008356c0, 0xc000835880, 0xc002148480, {0x0, 0x0}, {0x0, 0x0, ...}, ...})
test/e2e/storage/testsuites/provisioning.go:624 +0x808
k8s.io/kubernetes/test/e2e/storage/testsuites.(*provisioningTestSuite).DefineTests.func4()
test/e2e/storage/testsuites/provisioning.go:243 +0x618
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
�[1mSTEP�[0m: deleting the test namespace: provisioning-8483
�[1mSTEP�[0m: Waiting for namespaces [provisioning-8483] to vanish
May 25 06:39:59.332: INFO: error deleting namespace provisioning-8483: timed out waiting for the condition
�[1mSTEP�[0m: uninstalling csi gce-pd driver
May 25 06:39:59.333: INFO: deleting *v1.ServiceAccount: provisioning-8483-569/csi-attacher
May 25 06:39:59.426: INFO: deleting *v1.ClusterRole: external-attacher-runner-provisioning-8483
May 25 06:39:59.526: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-provisioning-8483
May 25 06:39:59.626: INFO: deleting *v1.Role: provisioning-8483-569/external-attacher-cfg-provisioning-8483
May 25 06:39:59.726: INFO: deleting *v1.RoleBinding: provisioning-8483-569/csi-attacher-role-cfg
May 25 06:39:59.826: INFO: deleting *v1.ServiceAccount: provisioning-8483-569/csi-provisioner
May 25 06:39:59.928: INFO: deleting *v1.ClusterRole: external-provisioner-runner-provisioning-8483
May 25 06:40:00.027: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-provisioning-8483
May 25 06:40:00.127: INFO: deleting *v1.Role: provisioning-8483-569/external-provisioner-cfg-provisioning-8483
May 25 06:40:00.227: INFO: deleting *v1.RoleBinding: provisioning-8483-569/csi-provisioner-role-cfg
May 25 06:40:00.326: INFO: deleting *v1.ServiceAccount: provisioning-8483-569/csi-gce-pd-controller-sa
May 25 06:40:00.426: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-provisioning-8483
May 25 06:40:00.538: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-provisioning-8483
May 25 06:40:00.627: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-provisioning-8483
May 25 06:40:00.733: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-provisioning-8483
May 25 06:40:00.826: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-provisioning-8483
May 25 06:40:00.925: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-provisioning-8483
May 25 06:40:01.027: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-provisioning-8483
May 25 06:40:01.128: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-provisioning-8483
May 25 06:40:01.226: INFO: deleting *v1.Role: provisioning-8483-569/csi-gce-pd-leaderelection-role-provisioning-8483
May 25 06:40:01.326: INFO: deleting *v1.RoleBinding: provisioning-8483-569/csi-gce-pd-controller-leaderelection-binding
May 25 06:40:01.428: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-provisioning-8483
May 25 06:40:01.536: INFO: deleting *v1.DaemonSet: provisioning-8483-569/csi-gce-pd-node
May 25 06:40:01.626: INFO: deleting *v1.StatefulSet: provisioning-8483-569/csi-gce-pd-controller
�[1mSTEP�[0m: deleting the driver namespace: provisioning-8483-569
�[1mSTEP�[0m: Waiting for namespaces [provisioning-8483-569] to vanish
[AfterEach] [Testpattern: Dynamic PV (default fs)] provisioning
test/e2e/framework/framework.go:187
�[1mSTEP�[0m: Collecting events from namespace "provisioning-8483".
�[1mSTEP�[0m: Found 0 events.
May 25 06:40:07.953: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 06:40:07.953: INFO:
May 25 06:40:07.996: INFO:
Logging node info for node bootstrap-e2e-master
May 25 06:40:08.039: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 10730 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 06:36:25 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:36:25 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:36:25 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:36:25 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:36:25 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:40:08.039: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 06:40:08.082: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 06:40:08.139: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container kube-apiserver ready: true, restart count 0
May 25 06:40:08.140: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container kube-scheduler ready: true, restart count 0
May 25 06:40:08.140: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container etcd-container ready: true, restart count 0
May 25 06:40:08.140: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 06:40:08.140: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container etcd-container ready: true, restart count 0
May 25 06:40:08.140: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 06:40:08.140: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 06:40:08.140: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 06:40:08.140: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:40:08.140: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:40:08.140: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.140: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 06:40:08.328: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 06:40:08.328: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 06:40:08.370: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 10725 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-gw73 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 06:19:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:35:18 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:38:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:38:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:38:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:38:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:38:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:35:18 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:35:18 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:35:18 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:35:18 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:40:08.370: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 06:40:08.413: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 06:40:08.479: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.479: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:40:08.479: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:40:08.479: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:40:08.479: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:40:08.479: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.479: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:40:08.479: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.479: INFO: Container coredns ready: true, restart count 0
May 25 06:40:08.660: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 06:40:08.660: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 06:40:08.702: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 10722 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-kn6c topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:18:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 06:38:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 06:39:36 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:38:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:38:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:38:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:38:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:39:36 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:39:36 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:39:36 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:39:36 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:40:08.703: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 06:40:08.746: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 06:40:08.819: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.819: INFO: Container autoscaler ready: true, restart count 0
May 25 06:40:08.819: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.819: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 06:40:08.819: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:40:08.819: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:40:08.819: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:40:08.819: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.819: INFO: Container coredns ready: true, restart count 0
May 25 06:40:08.819: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.819: INFO: Container default-http-backend ready: true, restart count 0
May 25 06:40:08.819: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.819: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:40:08.819: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:08.819: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:40:09.015: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 06:40:09.015: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 06:40:09.057: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 10720 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:36:39 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:38:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:38:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:38:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:38:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:38:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:38:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:36:39 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:36:39 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:36:39 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:36:39 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:40:09.058: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 06:40:09.101: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 06:40:09.164: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:09.164: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:40:09.164: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 06:40:09.164: INFO: Container metrics-server ready: true, restart count 0
May 25 06:40:09.164: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 06:40:09.164: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:40:09.164: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:40:09.164: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:40:09.164: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:40:09.164: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:40:09.335: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
May 25 06:40:09.336: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
�[1mSTEP�[0m: Destroying namespace "provisioning-8483" for this suite.
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sDynamic\sSnapshot\s\(delete\spolicy\)\]\ssnapshottable\[Feature\:VolumeSnapshotDataSource\]\svolume\ssnapshot\scontroller\s\sshould\scheck\ssnapshot\sfields\,\scheck\srestore\scorrectly\sworks\safter\smodifying\ssource\sdata\,\scheck\sdeletion\s\(persistent\)$'
test/e2e/storage/testsuites/snapshottable.go:278
May 25 05:50:41.461: Unexpected error:
<*errors.errorString | 0xc0035a60e0>: {
s: "VolumeSnapshot snapshot-2wsgn is not ready within 5m0s",
}
VolumeSnapshot snapshot-2wsgn is not ready within 5m0s
occurred
test/e2e/storage/utils/snapshot.go:86
[BeforeEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource]
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource]
test/e2e/framework/framework.go:186
�[1mSTEP�[0m: Creating a kubernetes client
May 25 05:44:58.230: INFO: >>> kubeConfig: /workspace/.kube/config
�[1mSTEP�[0m: Building a namespace api object, basename snapshotting
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
�[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace
[It] should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
test/e2e/storage/testsuites/snapshottable.go:278
�[1mSTEP�[0m: deploying csi gce-pd driver
�[1mSTEP�[0m: Building a driver namespace object, basename snapshotting-3720
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
May 25 05:44:58.866: INFO: Found CI service account key at /etc/service-account/service-account.json
May 25 05:44:58.866: INFO: Running cp [/etc/service-account/service-account.json /tmp/2913e779-b2da-4ab3-9374-43b7f032bd9d/cloud-sa.json]
May 25 05:44:58.911: INFO: Shredding file /tmp/2913e779-b2da-4ab3-9374-43b7f032bd9d/cloud-sa.json
May 25 05:44:58.911: INFO: Running shred [--remove /tmp/2913e779-b2da-4ab3-9374-43b7f032bd9d/cloud-sa.json]
May 25 05:44:58.945: INFO: File /tmp/2913e779-b2da-4ab3-9374-43b7f032bd9d/cloud-sa.json successfully shredded
May 25 05:44:58.952: INFO: creating *v1.ServiceAccount: snapshotting-3720-6758/csi-attacher
May 25 05:44:58.994: INFO: creating *v1.ClusterRole: external-attacher-runner-snapshotting-3720
May 25 05:44:58.994: INFO: Define cluster role external-attacher-runner-snapshotting-3720
May 25 05:44:59.036: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-3720
May 25 05:44:59.078: INFO: creating *v1.Role: snapshotting-3720-6758/external-attacher-cfg-snapshotting-3720
May 25 05:44:59.121: INFO: creating *v1.RoleBinding: snapshotting-3720-6758/csi-attacher-role-cfg
May 25 05:44:59.164: INFO: creating *v1.ServiceAccount: snapshotting-3720-6758/csi-provisioner
May 25 05:44:59.206: INFO: creating *v1.ClusterRole: external-provisioner-runner-snapshotting-3720
May 25 05:44:59.206: INFO: Define cluster role external-provisioner-runner-snapshotting-3720
May 25 05:44:59.249: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-3720
May 25 05:44:59.293: INFO: creating *v1.Role: snapshotting-3720-6758/external-provisioner-cfg-snapshotting-3720
May 25 05:44:59.336: INFO: creating *v1.RoleBinding: snapshotting-3720-6758/csi-provisioner-role-cfg
May 25 05:44:59.378: INFO: creating *v1.ServiceAccount: snapshotting-3720-6758/csi-gce-pd-controller-sa
May 25 05:44:59.421: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-3720
May 25 05:44:59.421: INFO: Define cluster role csi-gce-pd-provisioner-role-snapshotting-3720
May 25 05:44:59.463: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-3720
May 25 05:44:59.505: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-3720
May 25 05:44:59.505: INFO: Define cluster role csi-gce-pd-attacher-role-snapshotting-3720
May 25 05:44:59.547: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-3720
May 25 05:44:59.590: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-3720
May 25 05:44:59.590: INFO: Define cluster role csi-gce-pd-resizer-role-snapshotting-3720
May 25 05:44:59.632: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-3720
May 25 05:44:59.675: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-3720
May 25 05:44:59.675: INFO: Define cluster role csi-gce-pd-snapshotter-role-snapshotting-3720
May 25 05:44:59.719: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-3720
May 25 05:44:59.762: INFO: creating *v1.Role: snapshotting-3720-6758/csi-gce-pd-leaderelection-role-snapshotting-3720
May 25 05:44:59.804: INFO: creating *v1.RoleBinding: snapshotting-3720-6758/csi-gce-pd-controller-leaderelection-binding
May 25 05:44:59.846: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-3720
May 25 05:44:59.888: INFO: creating *v1.DaemonSet: snapshotting-3720-6758/csi-gce-pd-node
May 25 05:44:59.936: INFO: creating *v1.StatefulSet: snapshotting-3720-6758/csi-gce-pd-controller
May 25 05:45:00.182: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-gw73
May 25 05:45:09.951: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-kn6c
May 25 05:45:12.033: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-nndt
May 25 05:45:12.075: INFO: Creating resource for dynamic PV
May 25 05:45:12.075: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(pd.csi.storage.gke.io) supported size:{ 1Mi}
�[1mSTEP�[0m: creating a StorageClass snapshotting-3720rpjdw
�[1mSTEP�[0m: creating a claim
May 25 05:45:12.123: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
�[1mSTEP�[0m: [init] starting a pod to use the claim
�[1mSTEP�[0m: [init] check pod success
May 25 05:45:12.298: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-hqgzp" in namespace "snapshotting-3720" to be "Succeeded or Failed"
May 25 05:45:12.339: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 41.323076ms
May 25 05:45:14.382: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 2.084040314s
May 25 05:45:16.382: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 4.083824674s
May 25 05:45:18.382: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 6.083854464s
May 25 05:45:20.381: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 8.083346666s
May 25 05:45:22.381: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 10.082703368s
May 25 05:45:24.382: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 12.084080724s
May 25 05:45:26.382: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 14.083893524s
May 25 05:45:28.384: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 16.085878246s
May 25 05:45:30.382: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Pending", Reason="", readiness=false. Elapsed: 18.083702602s
May 25 05:45:32.383: INFO: Pod "pvc-snapshottable-tester-hqgzp": Phase="Succeeded", Reason="", readiness=false. Elapsed: 20.084701143s
�[1mSTEP�[0m: Saw pod success
May 25 05:45:32.383: INFO: Pod "pvc-snapshottable-tester-hqgzp" satisfied condition "Succeeded or Failed"
�[1mSTEP�[0m: [init] checking the claim
May 25 05:45:32.425: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pd.csi.storage.gke.io247s8] to have phase Bound
May 25 05:45:32.466: INFO: PersistentVolumeClaim pd.csi.storage.gke.io247s8 found and phase=Bound (41.051102ms)
�[1mSTEP�[0m: [init] checking the PV
�[1mSTEP�[0m: [init] deleting the pod
May 25 05:45:32.661: INFO: Pod pvc-snapshottable-tester-hqgzp has the following logs:
May 25 05:45:32.661: INFO: Deleting pod "pvc-snapshottable-tester-hqgzp" in namespace "snapshotting-3720"
May 25 05:45:32.713: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-hqgzp" to be fully deleted
�[1mSTEP�[0m: [init] waiting until the node=bootstrap-e2e-minion-group-gw73 is not using the volume=projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/disks/pvc-7070e031-ce6a-4015-afdf-577788d390e7
May 25 05:45:32.811: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/disks/pvc-7070e031-ce6a-4015-afdf-577788d390e7]
May 25 05:45:34.855: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/disks/pvc-7070e031-ce6a-4015-afdf-577788d390e7]
May 25 05:45:36.897: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/disks/pvc-7070e031-ce6a-4015-afdf-577788d390e7]
May 25 05:45:38.941: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/disks/pvc-7070e031-ce6a-4015-afdf-577788d390e7]
May 25 05:45:40.983: INFO: current volumes in use: []
May 25 05:45:40.983: INFO: WaitUntil finished successfully after 8.228085572s
�[1mSTEP�[0m: creating a SnapshotClass
�[1mSTEP�[0m: creating a dynamic VolumeSnapshot
May 25 05:45:41.115: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-2wsgn to become ready
May 25 05:45:41.164: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:43.206: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:45.249: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:47.291: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:49.333: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:51.376: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:53.419: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:55.462: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:57.505: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:45:59.547: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:01.590: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:03.632: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:05.675: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:07.719: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:09.762: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:11.804: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:13.848: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:15.892: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:17.942: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:19.985: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:22.027: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:24.069: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:26.112: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:28.156: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:30.199: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:32.242: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:34.284: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:36.332: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:38.375: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:40.419: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:42.462: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:44.504: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:46.547: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:48.588: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:50.632: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:52.696: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:54.738: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:56.782: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:46:58.825: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:00.868: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:02.911: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:04.954: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:06.996: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:09.039: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:11.081: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:13.124: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:15.166: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:17.209: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:19.252: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:21.295: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:23.337: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:25.381: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:27.423: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:29.466: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:31.508: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:33.552: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:35.594: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:37.637: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:39.679: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:41.721: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:43.763: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:45.806: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:47.850: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:49.895: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:51.942: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:53.985: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:56.027: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:47:58.069: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:00.112: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:02.155: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:04.197: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:06.239: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:08.282: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:10.324: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:12.367: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:14.410: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:16.452: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:18.494: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:20.538: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:22.580: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:24.623: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:26.667: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:28.709: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:30.753: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:32.796: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:34.838: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:36.882: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:38.924: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:40.966: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:43.009: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:45.052: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:47.095: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:49.138: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:51.180: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:53.222: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:55.266: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:57.308: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:48:59.349: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:01.392: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:03.436: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:05.478: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:07.521: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:09.563: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:11.606: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:13.648: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:15.692: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:17.736: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:19.778: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:21.836: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:23.879: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:25.921: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:27.964: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:30.007: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:32.049: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:34.092: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:36.134: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:38.176: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:40.218: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:42.263: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:44.306: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:46.349: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:48.392: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:50.435: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:52.478: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:54.519: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:56.561: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:49:58.604: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:00.646: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:02.689: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:04.731: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:06.774: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:08.824: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:10.866: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:12.908: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:14.950: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:16.993: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:19.036: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:21.079: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:23.122: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:25.165: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:27.207: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:29.250: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:31.292: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:33.334: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:35.376: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:37.418: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:39.460: INFO: VolumeSnapshot snapshot-2wsgn found but is not ready.
May 25 05:50:41.461: INFO: WaitUntil failed after reaching the timeout 5m0s
May 25 05:50:41.461: FAIL: Unexpected error:
<*errors.errorString | 0xc0035a60e0>: {
s: "VolumeSnapshot snapshot-2wsgn is not ready within 5m0s",
}
VolumeSnapshot snapshot-2wsgn is not ready within 5m0s
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eab80, 0xc0018cf7e8?}, 0xc00030ee68)
test/e2e/storage/utils/snapshot.go:86 +0x1ad
k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f220008b9b0, 0xc000569ee0}, 0xc0039c0f00, {{0x71ba8b5, 0x20}, {0x0, 0x0}, {0x7133ab2, 0x9}, {0x0, ...}, ...}, ...)
test/e2e/storage/framework/snapshot_resource.go:92 +0x246
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2()
test/e2e/storage/testsuites/snapshottable.go:359 +0xb3d
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
�[1mSTEP�[0m: checking the snapshot
�[1mSTEP�[0m: checking the SnapshotContent
�[1mSTEP�[0m: Modifying source data test
�[1mSTEP�[0m: modifying the data in the source PVC
May 25 05:50:41.594: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-4g22c" in namespace "snapshotting-3720" to be "Succeeded or Failed"
May 25 05:50:41.637: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Pending", Reason="", readiness=false. Elapsed: 43.504113ms
May 25 05:50:43.679: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.08537977s
May 25 05:50:45.678: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.084706049s
May 25 05:50:47.681: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.087320126s
May 25 05:50:49.680: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.086006623s
May 25 05:50:51.679: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.085323268s
May 25 05:50:53.679: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Pending", Reason="", readiness=false. Elapsed: 12.085498289s
May 25 05:50:55.678: INFO: Pod "pvc-snapshottable-data-tester-4g22c": Phase="Succeeded", Reason="", readiness=false. Elapsed: 14.08440291s
�[1mSTEP�[0m: Saw pod success
May 25 05:50:55.678: INFO: Pod "pvc-snapshottable-data-tester-4g22c" satisfied condition "Succeeded or Failed"
May 25 05:50:55.779: INFO: Pod pvc-snapshottable-data-tester-4g22c has the following logs:
May 25 05:50:55.779: INFO: Deleting pod "pvc-snapshottable-data-tester-4g22c" in namespace "snapshotting-3720"
May 25 05:50:55.829: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-4g22c" to be fully deleted
�[1mSTEP�[0m: creating a pvc from the snapshot
�[1mSTEP�[0m: starting a pod to use the snapshot
May 25 05:50:55.962: INFO: Waiting up to 15m0s for pod "restored-pvc-tester-vpgwt" in namespace "snapshotting-3720" to be "running"
May 25 05:50:56.003: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 40.561641ms
May 25 05:50:58.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.082910028s
May 25 05:51:00.046: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4.084031712s
May 25 05:51:02.046: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6.08412614s
May 25 05:51:04.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8.081837717s
May 25 05:51:06.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10.082318535s
May 25 05:51:08.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12.082107538s
May 25 05:51:10.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14.082975192s
May 25 05:51:12.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 16.082288518s
May 25 05:51:14.046: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 18.083767731s
May 25 05:51:16.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 20.082701732s
May 25 05:51:18.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 22.0824717s
May 25 05:51:20.046: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 24.083489945s
May 25 05:51:22.046: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 26.083835865s
May 25 05:51:24.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 28.082535507s
May 25 05:51:26.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 30.082253369s
May 25 05:51:28.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 32.083193734s
May 25 05:51:30.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 34.083089924s
May 25 05:51:32.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 36.08160331s
May 25 05:51:34.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 38.082903868s
May 25 05:51:36.048: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 40.085551507s
May 25 05:51:38.048: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 42.086404789s
May 25 05:51:40.046: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 44.083446748s
May 25 05:51:42.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 46.08256746s
May 25 05:51:44.049: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 48.086892096s
May 25 05:51:46.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 50.082819131s
May 25 05:51:48.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 52.082705408s
May 25 05:51:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 54.105291266s
May 25 05:51:52.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 56.104367028s
May 25 05:51:54.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 58.104488699s
May 25 05:51:56.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.10522073s
May 25 05:51:58.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.104553015s
May 25 05:52:00.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.104802661s
May 25 05:52:02.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.105024354s
May 25 05:52:04.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.104707027s
May 25 05:52:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.104111252s
May 25 05:52:08.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.104676574s
May 25 05:52:10.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.104332606s
May 25 05:52:12.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.104797967s
May 25 05:52:14.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.104638361s
May 25 05:52:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.104399117s
May 25 05:52:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.103826005s
May 25 05:52:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.105249448s
May 25 05:52:22.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.104454313s
May 25 05:52:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.103958486s
May 25 05:52:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.104114883s
May 25 05:52:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.104142833s
May 25 05:52:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.104961912s
May 25 05:52:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.104299762s
May 25 05:52:34.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.103761742s
May 25 05:52:36.070: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.107831675s
May 25 05:52:38.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.104350774s
May 25 05:52:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.105360812s
May 25 05:52:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.10383007s
May 25 05:52:44.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.104482343s
May 25 05:52:46.070: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.107428055s
May 25 05:52:48.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.10472812s
May 25 05:52:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.104507096s
May 25 05:52:52.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.10472002s
May 25 05:52:54.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.103814191s
May 25 05:52:56.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.104638594s
May 25 05:52:58.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.10410626s
May 25 05:53:00.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.105753952s
May 25 05:53:02.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.104468459s
May 25 05:53:04.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.10454925s
May 25 05:53:06.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.10515851s
May 25 05:53:08.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.104908491s
May 25 05:53:10.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.105822849s
May 25 05:53:12.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.10472276s
May 25 05:53:14.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.104632447s
May 25 05:53:16.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.104489061s
May 25 05:53:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.10401294s
May 25 05:53:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.10484177s
May 25 05:53:22.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.104566405s
May 25 05:53:24.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.104549488s
May 25 05:53:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.104064238s
May 25 05:53:28.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.105128037s
May 25 05:53:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.104566233s
May 25 05:53:32.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.105168735s
May 25 05:53:34.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.104632177s
May 25 05:53:36.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.104639204s
May 25 05:53:38.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.104288923s
May 25 05:53:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.104601039s
May 25 05:53:42.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.10475707s
May 25 05:53:44.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.103923317s
May 25 05:53:46.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.105167536s
May 25 05:53:48.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.104827233s
May 25 05:53:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.104516276s
May 25 05:53:52.099: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.136545461s
May 25 05:53:54.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.104496584s
May 25 05:53:56.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.104717351s
May 25 05:53:58.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.104841599s
May 25 05:54:00.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.105881716s
May 25 05:54:02.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.104253818s
May 25 05:54:04.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.104830901s
May 25 05:54:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.103933624s
May 25 05:54:08.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.104780198s
May 25 05:54:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.104421869s
May 25 05:54:12.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.103724685s
May 25 05:54:14.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.104250608s
May 25 05:54:16.077: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.114593328s
May 25 05:54:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.103774765s
May 25 05:54:20.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.104335271s
May 25 05:54:22.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.104561846s
May 25 05:54:24.074: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.111474073s
May 25 05:54:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.103878028s
May 25 05:54:28.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.105044249s
May 25 05:54:30.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.103879636s
May 25 05:54:32.070: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.108122653s
May 25 05:54:34.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.106537417s
May 25 05:54:36.077: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.114682575s
May 25 05:54:38.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.10649732s
May 25 05:54:40.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.105614308s
May 25 05:54:42.072: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.109888707s
May 25 05:54:44.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.106460892s
May 25 05:54:46.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.103659743s
May 25 05:54:48.073: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.110559816s
May 25 05:54:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.105403347s
May 25 05:54:52.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.082220936s
May 25 05:54:54.047: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.084438275s
May 25 05:54:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.103574228s
May 25 05:54:58.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.106582576s
May 25 05:55:00.073: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.110597553s
May 25 05:55:02.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.106546972s
May 25 05:55:04.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.106519569s
May 25 05:55:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.103962064s
May 25 05:55:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.103890609s
May 25 05:55:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.10447546s
May 25 05:55:12.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.106548507s
May 25 05:55:14.077: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.114673973s
May 25 05:55:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.103610343s
May 25 05:55:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.104213257s
May 25 05:55:20.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.104082163s
May 25 05:55:22.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.103877272s
May 25 05:55:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.103992844s
May 25 05:55:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.103682626s
May 25 05:55:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.103428395s
May 25 05:55:30.073: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.111181281s
May 25 05:55:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.103933467s
May 25 05:55:34.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.104270897s
May 25 05:55:36.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.103992614s
May 25 05:55:38.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.103665564s
May 25 05:55:40.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.103701057s
May 25 05:55:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.103516199s
May 25 05:55:44.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.103922831s
May 25 05:55:46.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.104014383s
May 25 05:55:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.103661391s
May 25 05:55:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.10454456s
May 25 05:55:52.104: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.141579622s
May 25 05:55:54.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.104143461s
May 25 05:55:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.103653696s
May 25 05:55:58.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m2.103271787s
May 25 05:56:00.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m4.104073237s
May 25 05:56:02.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m6.103755391s
May 25 05:56:04.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m8.104210846s
May 25 05:56:06.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m10.10334283s
May 25 05:56:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m12.103749588s
May 25 05:56:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m14.104976444s
May 25 05:56:12.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m16.104000657s
May 25 05:56:14.089: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m18.127108051s
May 25 05:56:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m20.103576183s
May 25 05:56:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m22.10422195s
May 25 05:56:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m24.105084s
May 25 05:56:22.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m26.104309921s
May 25 05:56:24.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m28.104535673s
May 25 05:56:26.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m30.081844072s
May 25 05:56:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m32.104228829s
May 25 05:56:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m34.104666424s
May 25 05:56:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m36.104268475s
May 25 05:56:34.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m38.104048343s
May 25 05:56:36.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m40.104078256s
May 25 05:56:38.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m42.104542672s
May 25 05:56:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m44.105191021s
May 25 05:56:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m46.10371762s
May 25 05:56:44.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m48.104679552s
May 25 05:56:46.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m50.104098949s
May 25 05:56:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m52.104093294s
May 25 05:56:50.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m54.103749845s
May 25 05:56:52.103: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m56.141344638s
May 25 05:56:54.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 5m58.104695422s
May 25 05:56:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m0.103537001s
May 25 05:56:58.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m2.103883901s
May 25 05:57:00.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m4.104542649s
May 25 05:57:02.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m6.106597908s
May 25 05:57:04.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m8.103948031s
May 25 05:57:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m10.104409711s
May 25 05:57:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m12.104293319s
May 25 05:57:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m14.104601993s
May 25 05:57:12.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m16.104138064s
May 25 05:57:14.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m18.103670415s
May 25 05:57:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m20.103578824s
May 25 05:57:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m22.104176342s
May 25 05:57:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m24.105148045s
May 25 05:57:22.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m26.104497725s
May 25 05:57:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m28.104413547s
May 25 05:57:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m30.104228857s
May 25 05:57:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m32.103574121s
May 25 05:57:30.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m34.104394698s
May 25 05:57:32.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m36.106618278s
May 25 05:57:34.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m38.103696057s
May 25 05:57:36.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m40.104384026s
May 25 05:57:38.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m42.103370941s
May 25 05:57:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m44.105290247s
May 25 05:57:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m46.103684793s
May 25 05:57:44.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m48.10339368s
May 25 05:57:46.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m50.103938916s
May 25 05:57:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m52.104075726s
May 25 05:57:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m54.10488629s
May 25 05:57:52.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m56.082574369s
May 25 05:57:54.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 6m58.104270141s
May 25 05:57:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m0.10399065s
May 25 05:57:58.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m2.103724313s
May 25 05:58:00.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m4.104152593s
May 25 05:58:02.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m6.103906046s
May 25 05:58:04.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m8.104321608s
May 25 05:58:06.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m10.104668847s
May 25 05:58:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m12.104044448s
May 25 05:58:10.070: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m14.107546769s
May 25 05:58:12.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m16.103599604s
May 25 05:58:14.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m18.10298105s
May 25 05:58:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m20.104175548s
May 25 05:58:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m22.103778639s
May 25 05:58:20.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m24.104071778s
May 25 05:58:22.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m26.104301097s
May 25 05:58:24.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m28.104511158s
May 25 05:58:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m30.103727004s
May 25 05:58:28.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m32.104474885s
May 25 05:58:30.071: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m34.108811914s
May 25 05:58:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m36.103972718s
May 25 05:58:34.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m38.104914342s
May 25 05:58:36.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m40.10339173s
May 25 05:58:38.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m42.082179022s
May 25 05:58:40.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m44.104350769s
May 25 05:58:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m46.103806063s
May 25 05:58:44.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m48.10417844s
May 25 05:58:46.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m50.103713809s
May 25 05:58:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m52.103923876s
May 25 05:58:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m54.105173234s
May 25 05:58:52.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m56.083393679s
May 25 05:58:54.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 7m58.103464955s
May 25 05:58:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m0.103799041s
May 25 05:58:58.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m2.104538191s
May 25 05:59:00.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m4.105384005s
May 25 05:59:02.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m6.104012868s
May 25 05:59:04.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m8.103726836s
May 25 05:59:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m10.104270759s
May 25 05:59:08.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m12.104524431s
May 25 05:59:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m14.104954819s
May 25 05:59:12.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m16.104925898s
May 25 05:59:14.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m18.104747607s
May 25 05:59:16.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m20.105626572s
May 25 05:59:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m22.104388036s
May 25 05:59:20.071: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m24.109084595s
May 25 05:59:22.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m26.103778011s
May 25 05:59:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m28.103444063s
May 25 05:59:26.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m30.104985882s
May 25 05:59:28.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m32.104905498s
May 25 05:59:30.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m34.104105104s
May 25 05:59:32.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m36.103256853s
May 25 05:59:34.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m38.105691551s
May 25 05:59:36.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m40.10376451s
May 25 05:59:38.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m42.104441042s
May 25 05:59:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m44.104621266s
May 25 05:59:42.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m46.104590098s
May 25 05:59:44.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m48.103731619s
May 25 05:59:46.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m50.103804251s
May 25 05:59:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m52.103644419s
May 25 05:59:50.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m54.104029467s
May 25 05:59:52.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m56.105608248s
May 25 05:59:54.109: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 8m58.146453965s
May 25 05:59:56.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m0.104498238s
May 25 05:59:58.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m2.10346589s
May 25 06:00:00.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m4.104432915s
May 25 06:00:02.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m6.104176926s
May 25 06:00:04.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m8.104043238s
May 25 06:00:06.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m10.104900778s
May 25 06:00:08.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m12.104671116s
May 25 06:00:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m14.105264585s
May 25 06:00:12.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m16.104494118s
May 25 06:00:14.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m18.104618242s
May 25 06:00:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m20.10388691s
May 25 06:00:18.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m22.104762616s
May 25 06:00:20.075: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m24.112853278s
May 25 06:00:22.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m26.10444266s
May 25 06:00:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m28.10363915s
May 25 06:00:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m30.103995276s
May 25 06:00:28.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m32.105583963s
May 25 06:00:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m34.104555968s
May 25 06:00:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m36.103783464s
May 25 06:00:34.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m38.104714249s
May 25 06:00:36.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m40.104344697s
May 25 06:00:38.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m42.106840009s
May 25 06:00:40.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m44.104364229s
May 25 06:00:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m46.103788044s
May 25 06:00:44.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m48.104463782s
May 25 06:00:46.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m50.104664295s
May 25 06:00:48.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m52.1050773s
May 25 06:00:50.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m54.106098542s
May 25 06:00:52.086: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m56.123977875s
May 25 06:00:54.084: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 9m58.122084596s
May 25 06:00:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m0.104142458s
May 25 06:00:58.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m2.104498964s
May 25 06:01:00.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m4.105468982s
May 25 06:01:02.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m6.104468902s
May 25 06:01:04.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m8.104817855s
May 25 06:01:06.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m10.106239147s
May 25 06:01:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m12.103824408s
May 25 06:01:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m14.104746302s
May 25 06:01:12.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m16.104646693s
May 25 06:01:14.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m18.104326865s
May 25 06:01:16.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m20.104484614s
May 25 06:01:18.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m22.104755663s
May 25 06:01:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m24.104882243s
May 25 06:01:22.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m26.103876037s
May 25 06:01:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m28.104260578s
May 25 06:01:26.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m30.104579199s
May 25 06:01:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m32.103888029s
May 25 06:01:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m34.104707514s
May 25 06:01:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m36.103438094s
May 25 06:01:34.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m38.104156771s
May 25 06:01:36.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m40.106012697s
May 25 06:01:38.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m42.103930354s
May 25 06:01:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m44.105195192s
May 25 06:01:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m46.104265481s
May 25 06:01:44.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m48.104089875s
May 25 06:01:46.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m50.104625088s
May 25 06:01:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m52.104124937s
May 25 06:01:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m54.10480627s
May 25 06:01:52.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m56.104355639s
May 25 06:01:54.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 10m58.105379025s
May 25 06:01:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m0.104346373s
May 25 06:01:58.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m2.104285463s
May 25 06:02:00.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m4.105073214s
May 25 06:02:02.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m6.10462962s
May 25 06:02:04.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m8.10464221s
May 25 06:02:06.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m10.08224458s
May 25 06:02:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m12.103830825s
May 25 06:02:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m14.104851361s
May 25 06:02:12.073: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m16.110529962s
May 25 06:02:14.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m18.104373577s
May 25 06:02:16.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m20.082671317s
May 25 06:02:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m22.10372806s
May 25 06:02:20.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m24.082959907s
May 25 06:02:22.080: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m26.118108994s
May 25 06:02:24.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m28.104703052s
May 25 06:02:26.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m30.082169281s
May 25 06:02:28.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m32.105200974s
May 25 06:02:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m34.104612874s
May 25 06:02:32.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m36.104555223s
May 25 06:02:34.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m38.104685099s
May 25 06:02:36.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m40.104688842s
May 25 06:02:38.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m42.103870936s
May 25 06:02:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m44.104616532s
May 25 06:02:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m46.104028458s
May 25 06:02:44.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m48.10463132s
May 25 06:02:46.044: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m50.082095845s
May 25 06:02:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m52.103920585s
May 25 06:02:50.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m54.103947688s
May 25 06:02:52.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m56.107068699s
May 25 06:02:54.072: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 11m58.109956786s
May 25 06:02:56.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m0.104481026s
May 25 06:02:58.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m2.104368931s
May 25 06:03:00.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m4.105100714s
May 25 06:03:02.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m6.104642057s
May 25 06:03:04.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m8.104344897s
May 25 06:03:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m10.104191927s
May 25 06:03:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m12.10364968s
May 25 06:03:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m14.104695887s
May 25 06:03:12.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m16.104648105s
May 25 06:03:14.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m18.104659501s
May 25 06:03:16.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m20.104453239s
May 25 06:03:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m22.104024572s
May 25 06:03:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m24.104574538s
May 25 06:03:22.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m26.104171908s
May 25 06:03:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m28.104403932s
May 25 06:03:26.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m30.106366635s
May 25 06:03:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m32.103729571s
May 25 06:03:30.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m34.104416443s
May 25 06:03:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m36.103598774s
May 25 06:03:34.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m38.10420905s
May 25 06:03:36.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m40.104423872s
May 25 06:03:38.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m42.104337831s
May 25 06:03:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m44.104659259s
May 25 06:03:42.069: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m46.106943s
May 25 06:03:44.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m48.103761391s
May 25 06:03:46.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m50.103975477s
May 25 06:03:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m52.103944435s
May 25 06:03:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m54.104703262s
May 25 06:03:52.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m56.105173255s
May 25 06:03:54.076: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 12m58.114015987s
May 25 06:03:56.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m0.104074262s
May 25 06:03:58.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m2.10487475s
May 25 06:04:00.070: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m4.107956545s
May 25 06:04:02.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m6.105065624s
May 25 06:04:04.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m8.104063673s
May 25 06:04:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m10.104082171s
May 25 06:04:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m12.103713121s
May 25 06:04:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m14.105073029s
May 25 06:04:12.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m16.10464091s
May 25 06:04:14.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m18.104155537s
May 25 06:04:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m20.10426831s
May 25 06:04:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m22.104337907s
May 25 06:04:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m24.104783413s
May 25 06:04:22.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m26.103823105s
May 25 06:04:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m28.104029842s
May 25 06:04:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m30.103510602s
May 25 06:04:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m32.103653778s
May 25 06:04:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m34.104626541s
May 25 06:04:32.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m36.104876355s
May 25 06:04:34.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m38.103861832s
May 25 06:04:36.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m40.104680012s
May 25 06:04:38.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m42.10395438s
May 25 06:04:40.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m44.106357611s
May 25 06:04:42.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m46.104300846s
May 25 06:04:44.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m48.104019818s
May 25 06:04:46.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m50.104430732s
May 25 06:04:48.055: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m52.0930351s
May 25 06:04:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m54.104920425s
May 25 06:04:52.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m56.104693616s
May 25 06:04:54.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 13m58.104262228s
May 25 06:04:56.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m0.104508199s
May 25 06:04:58.065: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m2.103192231s
May 25 06:05:00.068: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m4.105467838s
May 25 06:05:02.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m6.103764944s
May 25 06:05:04.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m8.104598335s
May 25 06:05:06.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m10.103963904s
May 25 06:05:08.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m12.104264562s
May 25 06:05:10.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m14.104601362s
May 25 06:05:12.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m16.104015669s
May 25 06:05:14.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m18.104037586s
May 25 06:05:16.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m20.104174297s
May 25 06:05:18.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m22.10342937s
May 25 06:05:20.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m24.104880749s
May 25 06:05:22.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m26.104593332s
May 25 06:05:24.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m28.103664644s
May 25 06:05:26.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m30.103723984s
May 25 06:05:28.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m32.103440355s
May 25 06:05:30.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m34.104742337s
May 25 06:05:32.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m36.104279189s
May 25 06:05:34.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m38.10469603s
May 25 06:05:36.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m40.104056084s
May 25 06:05:38.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m42.104809225s
May 25 06:05:40.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m44.104522565s
May 25 06:05:42.070: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m46.107787728s
May 25 06:05:44.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m48.104744818s
May 25 06:05:46.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.104679415s
May 25 06:05:48.066: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.104155105s
May 25 06:05:50.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.104803814s
May 25 06:05:52.067: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.10491544s
May 25 06:05:54.045: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.082543214s
May 25 06:05:56.070: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.108293932s
May 25 06:05:56.167: INFO: Pod "restored-pvc-tester-vpgwt": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.20463244s
May 25 06:05:56.167: INFO: Timed out while waiting for pod snapshotting-3720/restored-pvc-tester-vpgwt to be running. Last observed as: <*v1.Pod | 0xc0034efc00>: {
TypeMeta: {Kind: "", APIVersion: ""},
ObjectMeta: {
Name: "restored-pvc-tester-vpgwt",
GenerateName: "restored-pvc-tester-",
Namespace: "snapshotting-3720",
SelfLink: "",
UID: "63dee03c-a39c-4403-865f-8e0780fa56cd",
ResourceVersion: "3673",
Generation: 0,
CreationTimestamp: {
Time: 2022-05-25T05:50:55Z,
},
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: {
"app": "restored-pvc-tester",
},
Annotations: nil,
OwnerReferences: nil,
Finalizers: nil,
ManagedFields: [
{
Manager: "e2e.test",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: 2022-05-25T05:50:55Z,
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:metadata\":{\"f:generateName\":{},\"f:labels\":{\".\":{},\"f:app\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"volume-tester\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/mnt/test\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"my-volume\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}",
},
Subresource: "",
},
{
Manager: "kube-scheduler",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: 2022-05-25T06:00:55Z,
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:status\":{\"f:conditions\":{\".\":{},\"k:{\\\"type\\\":\\\"PodScheduled\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}}}}",
},
Subresource: "status",
},
],
},
Spec: {
Volumes: [
{
Name: "my-volume",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: {
ClaimName: "pvc-8v89c",
ReadOnly: false,
},
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: nil,
PortworxVolume: nil,
ScaleIO: nil,
StorageOS: nil,
CSI: nil,
Ephemeral: nil,
},
},
{
Name: "kube-api-access-6xwf5",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: {
Sources: [
{
Secret: nil,
DownwardAPI: nil,
ConfigMap: nil,
ServiceAccountToken: {
Audience: "",
ExpirationSeconds: 3607,
Path: "token",
},
},
{
Secret: nil,
DownwardAPI: nil,
ConfigMap: {
LocalObjectReference: {Name: ...},
Items: [...],
Optional: nil,
},
ServiceAccountToken: nil,
},
{
Secret: nil,
DownwardAPI: {Items: [...]},
ConfigMap: nil,
ServiceAccountToken: nil,
},
],
DefaultMode: 420,
},
PortworxVolume: nil,
ScaleIO: nil,
StorageOS: nil,
CSI: nil,
Ephemeral: nil,
},
},
],
InitContainers: nil,
Containers: [
{
Name: "volume-tester",
Image: "k8s.gcr.io/e2e-test-images/busybox:1.29-2",
Command: ["/bin/sh", "-c", "sleep 300"],
Args: nil,
WorkingDir: "",
Ports: nil,
EnvFrom: nil,
Env: nil,
Resources: {Limits: nil, Requests: nil},
VolumeMounts: [
{
Name: "my-volume",
ReadOnly: false,
MountPath: "/mnt/test",
SubPath: "",
MountPropagation: nil,
SubPathExpr: "",
},
{
Name: "kube-api-access-6xwf5",
ReadOnly: true,
MountPath: "/var/run/secrets/kubernetes.io/serviceaccount",
SubPath: "",
MountPropagation: nil,
SubPathExpr: "",
},
],
VolumeDevices: nil,
LivenessProbe: nil,
ReadinessProbe: nil,
StartupProbe: nil,
Lifecycle: nil,
TerminationMessagePath: "/dev/termination-log",
TerminationMessagePolicy: "File",
ImagePullPolicy: "IfNotPresent",
SecurityContext: nil,
Stdin: false,
StdinOnce: false,
TTY: false,
},
],
EphemeralContainers: nil,
RestartPolicy: "Never",
TerminationGracePeriodSeconds: 30,
ActiveDeadlineSeconds: nil,
DNSPolicy: "ClusterFirst",
NodeSelector: nil,
ServiceAccountName: "default",
DeprecatedServiceAccount: "default",
AutomountServiceAccountToken: nil,
NodeName: "",
HostNetwork: false,
HostPID: false,
HostIPC: false,
ShareProcessNamespace: nil,
SecurityContext: {
SELinuxOptions: nil,
WindowsOptions: nil,
RunAsUser: nil,
RunAsGroup: nil,
RunAsNonRoot: nil,
SupplementalGroups: nil,
FSGroup: nil,
Sysctls: nil,
FSGroupChangePolicy: nil,
SeccompProfile: nil,
},
ImagePullSecrets: nil,
Hostname: "",
Subdomain: "",
Affinity: nil,
SchedulerName: "default-scheduler",
Tolerations: [
{
Key: "node.kubernetes.io/not-ready",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: 300,
},
{
Key: "node.kubernetes.io/unreachable",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: 300,
},
],
HostAliases: nil,
PriorityClassName: "",
Priority: 0,
DNSConfig: nil,
ReadinessGates: nil,
RuntimeClassName: nil,
EnableServiceLinks: true,
PreemptionPolicy: "PreemptLowerPriority",
Overhead: nil,
TopologySpreadConstraints: nil,
SetHostnameAsFQDN: nil,
OS: nil,
},
Status: {
Phase: "Pending",
Conditions: [
{
Type: "PodScheduled",
Status: "False",
LastProbeTime: {
Time: 0001-01-01T00:00:00Z,
},
LastTransitionTime: {
Time: 2022-05-25T06:00:55Z,
},
Reason: "SchedulerError",
Message: "running PreBind plugin \"VolumeBinding\": binding volumes: timed out waiting for the condition",
},
],
Message: "",
Reason: "",
NominatedNodeName: "",
HostIP: "",
PodIP: "",
PodIPs: nil,
StartTime: nil,
InitContainerStatuses: nil,
ContainerStatuses: nil,
QOSClass: "BestEffort",
EphemeralContainerStatuses: nil,
},
}
May 25 06:05:56.167: FAIL: Unexpected error:
<*pod.timeoutError | 0xc00142fb40>: {
msg: "timed out while waiting for pod snapshotting-3720/restored-pvc-tester-vpgwt to be running",
}
timed out while waiting for pod snapshotting-3720/restored-pvc-tester-vpgwt to be running
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2()
test/e2e/storage/testsuites/snapshottable.go:410 +0x1685
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
[AfterEach] volume snapshot controller
test/e2e/storage/testsuites/snapshottable.go:172
May 25 06:05:56.266: INFO: Pod restored-pvc-tester-vpgwt has the following logs:
May 25 06:05:56.266: INFO: Deleting pod "restored-pvc-tester-vpgwt" in namespace "snapshotting-3720"
May 25 06:05:56.373: INFO: Wait up to 5m0s for pod "restored-pvc-tester-vpgwt" to be fully deleted
May 25 06:05:56.467: INFO: deleting claim "snapshotting-3720"/"pvc-8v89c"
May 25 06:05:56.570: INFO: deleting snapshot "snapshotting-3720"/"snapshot-2wsgn"
May 25 06:05:56.612: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 readyToUse:false]
May 25 06:05:56.612: INFO: snapshotContentName snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92
May 25 06:05:56.698: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 to be deleted
May 25 06:05:56.746: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:05:58.788: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:00.830: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:02.871: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:04.914: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:06.957: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:08.999: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:11.043: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:13.085: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:15.140: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:17.184: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:19.228: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:21.275: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:23.318: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:25.361: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:27.403: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:29.447: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:31.490: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:33.532: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:35.574: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:37.622: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:39.664: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:41.706: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:43.749: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:45.791: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:47.835: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:49.878: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:51.920: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:53.963: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:56.005: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:06:58.047: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:00.089: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:02.131: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:04.174: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:06.217: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:08.261: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:10.303: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:12.345: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:14.388: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:16.430: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:18.471: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:20.512: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:22.556: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:24.599: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:26.641: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:28.685: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:30.728: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:32.771: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:34.813: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:36.856: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:38.898: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:40.941: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:42.984: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:45.027: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:47.070: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:49.112: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:51.154: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:53.198: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:55.241: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:57.284: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:07:59.326: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:01.370: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:03.413: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:05.456: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:07.505: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:09.550: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:11.592: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:13.633: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:15.678: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:17.759: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:19.802: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:21.849: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:23.891: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:25.934: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:27.976: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:30.020: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:32.085: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:34.127: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:36.169: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:38.211: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:40.259: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:42.302: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:44.344: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:46.387: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:48.430: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:50.472: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:52.528: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:54.571: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:56.613: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:08:58.658: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:00.700: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:02.747: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:04.789: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:06.831: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:08.873: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:10.919: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:12.978: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:15.021: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:17.062: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:19.105: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:21.147: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:23.189: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:25.235: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:27.276: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:29.320: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:31.363: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:33.420: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:35.462: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:37.504: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:39.547: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:41.590: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:43.632: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:45.674: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:47.721: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:49.762: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:51.806: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:53.914: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:55.956: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:09:57.998: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:00.042: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:02.085: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:04.149: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:06.191: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:08.234: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:10.299: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:12.342: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:14.385: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:16.428: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:18.470: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:20.513: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:22.558: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:24.600: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:26.643: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:28.685: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:30.728: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:32.772: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:34.814: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:36.858: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:38.901: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:40.943: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:43.018: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:45.060: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:47.104: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:49.147: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:51.190: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:53.232: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:55.281: INFO: volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 has been found and is not deleted
May 25 06:10:57.282: INFO: WaitUntil failed after reaching the timeout 5m0s
May 25 06:10:57.283: FAIL: Unexpected error:
<*errors.errorString | 0xc001f0e9d0>: {
s: "volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 is not deleted within 5m0s",
}
volumesnapshotcontents snapcontent-8199d352-114f-41a7-90ec-8e3c365eec92 is not deleted within 5m0s
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/framework.(*SnapshotResource).CleanupResource(0xc000bd4fd0, 0xc000fc1d80)
test/e2e/storage/framework/snapshot_resource.go:204 +0xefd
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2.2()
test/e2e/storage/testsuites/snapshottable.go:361 +0x28
k8s.io/kubernetes/test/e2e/storage/utils.TryFunc(0xc00048c3c0?)
test/e2e/storage/utils/utils.go:714 +0x6d
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.2()
test/e2e/storage/testsuites/snapshottable.go:167 +0x38
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.3()
test/e2e/storage/testsuites/snapshottable.go:173 +0x1b
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
�[1mSTEP�[0m: Deleting pvc
May 25 06:10:57.466: INFO: Deleting PersistentVolumeClaim "pd.csi.storage.gke.io247s8"
May 25 06:10:57.568: INFO: Waiting up to 5m0s for PersistentVolume pvc-7070e031-ce6a-4015-afdf-577788d390e7 to get deleted
May 25 06:10:57.667: INFO: PersistentVolume pvc-7070e031-ce6a-4015-afdf-577788d390e7 found and phase=Released (98.497188ms)
May 25 06:11:02.717: INFO: PersistentVolume pvc-7070e031-ce6a-4015-afdf-577788d390e7 was removed
�[1mSTEP�[0m: Deleting sc
�[1mSTEP�[0m: deleting the test namespace: snapshotting-3720
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-3720".
�[1mSTEP�[0m: Found 22 events.
May 25 06:11:02.918: INFO: At 2022-05-25 05:45:11 +0000 UTC - event for pd.csi.storage.gke.io247s8: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
May 25 06:11:02.918: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for pd.csi.storage.gke.io247s8: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
May 25 06:11:02.918: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for pd.csi.storage.gke.io247s8: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_35efd7be-cb38-4041-8c6f-0e023e87e501 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-3720/pd.csi.storage.gke.io247s8"
May 25 06:11:02.918: INFO: At 2022-05-25 05:45:16 +0000 UTC - event for pd.csi.storage.gke.io247s8: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_35efd7be-cb38-4041-8c6f-0e023e87e501 } ProvisioningSucceeded: Successfully provisioned volume pvc-7070e031-ce6a-4015-afdf-577788d390e7
May 25 06:11:02.918: INFO: At 2022-05-25 05:45:17 +0000 UTC - event for pvc-snapshottable-tester-hqgzp: {default-scheduler } Scheduled: Successfully assigned snapshotting-3720/pvc-snapshottable-tester-hqgzp to bootstrap-e2e-minion-group-gw73
May 25 06:11:02.918: INFO: At 2022-05-25 05:45:26 +0000 UTC - event for pvc-snapshottable-tester-hqgzp: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-7070e031-ce6a-4015-afdf-577788d390e7"
May 25 06:11:02.918: INFO: At 2022-05-25 05:45:27 +0000 UTC - event for pvc-snapshottable-tester-hqgzp: {kubelet bootstrap-e2e-minion-group-gw73} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-2"
May 25 06:11:02.919: INFO: At 2022-05-25 05:45:28 +0000 UTC - event for pvc-snapshottable-tester-hqgzp: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container volume-tester
May 25 06:11:02.919: INFO: At 2022-05-25 05:45:28 +0000 UTC - event for pvc-snapshottable-tester-hqgzp: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container volume-tester
May 25 06:11:02.919: INFO: At 2022-05-25 05:45:28 +0000 UTC - event for pvc-snapshottable-tester-hqgzp: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" in 778.536983ms
May 25 06:11:02.919: INFO: At 2022-05-25 05:45:41 +0000 UTC - event for snapshot-2wsgn: {snapshot-controller } CreatingSnapshot: Waiting for a snapshot snapshotting-3720/snapshot-2wsgn to be created by the CSI driver.
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:41 +0000 UTC - event for pvc-snapshottable-data-tester-4g22c: {default-scheduler } Scheduled: Successfully assigned snapshotting-3720/pvc-snapshottable-data-tester-4g22c to bootstrap-e2e-minion-group-gw73
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:50 +0000 UTC - event for pvc-snapshottable-data-tester-4g22c: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-7070e031-ce6a-4015-afdf-577788d390e7"
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:52 +0000 UTC - event for pvc-snapshottable-data-tester-4g22c: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" already present on machine
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:52 +0000 UTC - event for pvc-snapshottable-data-tester-4g22c: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container volume-tester
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:52 +0000 UTC - event for pvc-snapshottable-data-tester-4g22c: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container volume-tester
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:55 +0000 UTC - event for pvc-8v89c: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_35efd7be-cb38-4041-8c6f-0e023e87e501 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-3720/pvc-8v89c"
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:55 +0000 UTC - event for pvc-8v89c: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:55 +0000 UTC - event for pvc-8v89c: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
May 25 06:11:02.919: INFO: At 2022-05-25 05:50:55 +0000 UTC - event for pvc-8v89c: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_35efd7be-cb38-4041-8c6f-0e023e87e501 } ProvisioningFailed: failed to provision volume with StorageClass "snapshotting-3720rpjdw": error getting handle for DataSource Type VolumeSnapshot by Name snapshot-2wsgn: error getting snapshot snapshot-2wsgn from api server: the server could not find the requested resource (get volumesnapshots.snapshot.storage.k8s.io snapshot-2wsgn)
May 25 06:11:02.919: INFO: At 2022-05-25 06:00:55 +0000 UTC - event for restored-pvc-tester-vpgwt: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
May 25 06:11:02.919: INFO: At 2022-05-25 06:05:57 +0000 UTC - event for restored-pvc-tester-vpgwt: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "restored-pvc-tester-vpgwt" not found
May 25 06:11:03.016: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 06:11:03.016: INFO:
May 25 06:11:03.118: INFO:
Logging node info for node bootstrap-e2e-master
May 25 06:11:03.217: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 5361 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-master"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 06:10:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:10:55 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:10:55 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:10:55 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:10:55 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:11:03.218: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 06:11:03.318: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 06:11:03.444: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:03.444: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:11:03.444: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:11:03.444: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 06:11:03.444: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 06:11:03.444: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container etcd-container ready: true, restart count 0
May 25 06:11:03.444: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 06:11:03.444: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 06:11:03.444: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container etcd-container ready: true, restart count 0
May 25 06:11:03.444: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container kube-apiserver ready: true, restart count 0
May 25 06:11:03.444: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:03.444: INFO: Container kube-scheduler ready: true, restart count 0
May 25 06:11:03.444: INFO: csi-gce-pd-node-fjwm7 started at 2022-05-25 05:44:59 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:03.444: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 06:11:03.444: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 06:11:03.766: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 06:11:03.766: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 06:11:03.817: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 4910 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-gw73"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 05:50:50 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:06:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:08:13 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:06:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:11:03.818: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 06:11:03.918: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 06:11:04.050: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:04.050: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:11:04.050: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:11:04.050: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.050: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:11:04.050: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.050: INFO: Container coredns ready: true, restart count 0
May 25 06:11:04.050: INFO: csi-gce-pd-node-bpxgm started at 2022-05-25 05:45:00 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:04.050: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 06:11:04.050: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 06:11:04.050: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.050: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:11:04.358: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 06:11:04.358: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 06:11:04.417: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 4911 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-kn6c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:06:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:08:13 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:04 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:04 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:04 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:06:04 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:11:04.420: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 06:11:04.519: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 06:11:04.637: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.637: INFO: Container coredns ready: true, restart count 0
May 25 06:11:04.637: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.637: INFO: Container autoscaler ready: true, restart count 0
May 25 06:11:04.637: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.637: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 06:11:04.637: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.637: INFO: Container default-http-backend ready: true, restart count 0
May 25 06:11:04.637: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.637: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:11:04.637: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:04.637: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:11:04.637: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:04.637: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:11:04.637: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:11:04.637: INFO: csi-gce-pd-node-q6qhv started at 2022-05-25 05:45:00 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:04.637: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 06:11:04.637: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 06:11:04.963: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 06:11:04.963: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 06:11:05.023: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 4915 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-nndt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:06:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:08:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:08:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:08:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:08:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:08:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:08:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:01 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:01 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:06:01 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:06:01 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:11:05.024: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 06:11:05.130: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 06:11:05.295: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:05.296: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:11:05.296: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:05.296: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:11:05.297: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:11:05.297: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 06:11:05.297: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:11:05.297: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:05.297: INFO: Container metrics-server ready: true, restart count 0
May 25 06:11:05.297: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 06:11:05.297: INFO: csi-gce-pd-node-h424b started at 2022-05-25 05:44:59 +0000 UTC (0+2 container statuses recorded)
May 25 06:11:05.297: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 06:11:05.297: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 06:11:05.297: INFO: csi-gce-pd-controller-0 started at 2022-05-25 05:45:00 +0000 UTC (0+5 container statuses recorded)
May 25 06:11:05.297: INFO: Container csi-attacher ready: true, restart count 0
May 25 06:11:05.297: INFO: Container csi-provisioner ready: true, restart count 0
May 25 06:11:05.297: INFO: Container csi-resizer ready: true, restart count 0
May 25 06:11:05.297: INFO: Container csi-snapshotter ready: true, restart count 0
May 25 06:11:05.297: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 06:11:05.631: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-3720] to vanish
May 25 06:16:05.866: INFO: error deleting namespace snapshotting-3720: timed out waiting for the condition
�[1mSTEP�[0m: uninstalling csi gce-pd driver
May 25 06:16:05.867: INFO: deleting *v1.ServiceAccount: snapshotting-3720-6758/csi-attacher
May 25 06:16:05.967: INFO: deleting *v1.ClusterRole: external-attacher-runner-snapshotting-3720
May 25 06:16:06.067: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-3720
May 25 06:16:06.167: INFO: deleting *v1.Role: snapshotting-3720-6758/external-attacher-cfg-snapshotting-3720
May 25 06:16:06.267: INFO: deleting *v1.RoleBinding: snapshotting-3720-6758/csi-attacher-role-cfg
May 25 06:16:06.367: INFO: deleting *v1.ServiceAccount: snapshotting-3720-6758/csi-provisioner
May 25 06:16:06.468: INFO: deleting *v1.ClusterRole: external-provisioner-runner-snapshotting-3720
May 25 06:16:06.568: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-3720
May 25 06:16:06.667: INFO: deleting *v1.Role: snapshotting-3720-6758/external-provisioner-cfg-snapshotting-3720
May 25 06:16:06.768: INFO: deleting *v1.RoleBinding: snapshotting-3720-6758/csi-provisioner-role-cfg
May 25 06:16:06.866: INFO: deleting *v1.ServiceAccount: snapshotting-3720-6758/csi-gce-pd-controller-sa
May 25 06:16:06.966: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-3720
May 25 06:16:07.067: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-3720
May 25 06:16:07.168: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-3720
May 25 06:16:07.268: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-3720
May 25 06:16:07.367: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-3720
May 25 06:16:07.468: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-3720
May 25 06:16:07.567: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-3720
May 25 06:16:07.667: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-3720
May 25 06:16:07.768: INFO: deleting *v1.Role: snapshotting-3720-6758/csi-gce-pd-leaderelection-role-snapshotting-3720
May 25 06:16:07.871: INFO: deleting *v1.RoleBinding: snapshotting-3720-6758/csi-gce-pd-controller-leaderelection-binding
May 25 06:16:07.967: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-3720
May 25 06:16:08.066: INFO: deleting *v1.DaemonSet: snapshotting-3720-6758/csi-gce-pd-node
May 25 06:16:08.167: INFO: deleting *v1.StatefulSet: snapshotting-3720-6758/csi-gce-pd-controller
�[1mSTEP�[0m: deleting the driver namespace: snapshotting-3720-6758
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-3720-6758".
�[1mSTEP�[0m: Found 82 events.
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-controller: {statefulset-controller } SuccessfulCreate: create Pod csi-gce-pd-controller-0 in StatefulSet csi-gce-pd-controller successful
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-fjwm7
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-h424b
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-bpxgm
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-q6qhv
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node-bpxgm: {default-scheduler } Scheduled: Successfully assigned snapshotting-3720-6758/csi-gce-pd-node-bpxgm to bootstrap-e2e-minion-group-gw73
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node-fjwm7: {default-scheduler } Scheduled: Successfully assigned snapshotting-3720-6758/csi-gce-pd-node-fjwm7 to bootstrap-e2e-master
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node-h424b: {default-scheduler } Scheduled: Successfully assigned snapshotting-3720-6758/csi-gce-pd-node-h424b to bootstrap-e2e-minion-group-nndt
May 25 06:16:08.369: INFO: At 2022-05-25 05:44:59 +0000 UTC - event for csi-gce-pd-node-q6qhv: {default-scheduler } Scheduled: Successfully assigned snapshotting-3720-6758/csi-gce-pd-node-q6qhv to bootstrap-e2e-minion-group-kn6c
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:00 +0000 UTC - event for csi-gce-pd-controller-0: {default-scheduler } Scheduled: Successfully assigned snapshotting-3720-6758/csi-gce-pd-controller-0 to bootstrap-e2e-minion-group-nndt
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:00 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:00 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:00 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:01 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:01 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-z592b" : failed to sync configmap cache: timed out waiting for the condition
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 1.288554046s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 1.592408889s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Created: Created container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 1.284723817s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:02 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:03 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" in 1.894512254s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:03 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Started: Started container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:03 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:03 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:03 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:03 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Pulling: Pulling image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:03 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" in 942.574779ms
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:04 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:04 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-snapshotter
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:04 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-snapshotter
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:06 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0" in 2.001011804s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:06 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 4.045262072s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:07 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 4.141994249s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-provisioner
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-provisioner
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:08 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 6.164905581s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:09 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Pulled: Successfully pulled image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" in 6.838210117s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:09 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:09 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:10 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-attacher
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:10 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0"
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:10 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-attacher
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:10 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0" in 1.523176082s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:10 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Started: Started container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:10 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Created: Created container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:11 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0" in 1.482931719s
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-resizer
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-resizer
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for external-snapshotter-leader-pd-csi-storage-gke-io: {external-snapshotter-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:12 +0000 UTC - event for pd-csi-storage-gke-io: {pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:13 +0000 UTC - event for external-attacher-leader-pd-csi-storage-gke-io: {external-attacher-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 06:16:08.369: INFO: At 2022-05-25 05:45:13 +0000 UTC - event for external-resizer-pd-csi-storage-gke-io: {external-resizer-pd-csi-storage-gke-io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-provisioner
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-attacher
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-resizer
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-snapshotter
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Killing: Stopping container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} Killing: Stopping container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-bpxgm: {kubelet bootstrap-e2e-minion-group-gw73} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-bpxgm_snapshotting-3720-6758(bbfcc8a2-1ef9-4064-8b0e-04020dc1334d)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "60bf9835fd7eca4e4e4be6e5eb1e2009aec76f79063f48fc6a19c89e402bc772": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Killing: Stopping container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-fjwm7: {kubelet bootstrap-e2e-master} Killing: Stopping container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-h424b: {kubelet bootstrap-e2e-minion-group-nndt} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-h424b_snapshotting-3720-6758(5fef3b7e-96f0-4b8d-ab51-882067d846ce)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "cca2bdc905989681563b06f75fe88d0b1f6227eca69c4475735f06b1b02aae1f": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Killing: Stopping container csi-driver-registrar
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} Killing: Stopping container gce-pd-driver
May 25 06:16:08.369: INFO: At 2022-05-25 06:16:08 +0000 UTC - event for csi-gce-pd-node-q6qhv: {kubelet bootstrap-e2e-minion-group-kn6c} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-q6qhv_snapshotting-3720-6758(ab4fd65f-0a50-48f6-b9c3-ba8f7d70a3e3)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "b141bfcc476aeecbb165cf938b60a6873ac5c87085796a60795da65363788057": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 06:16:08.470: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 06:16:08.470: INFO: csi-gce-pd-controller-0 bootstrap-e2e-minion-group-nndt Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:00 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:13 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:13 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:44:59 +0000 UTC }]
May 25 06:16:08.470: INFO: csi-gce-pd-node-bpxgm bootstrap-e2e-minion-group-gw73 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:00 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:08 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:08 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:44:59 +0000 UTC }]
May 25 06:16:08.470: INFO: csi-gce-pd-node-fjwm7 bootstrap-e2e-master Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:44:59 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:11 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:11 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:44:59 +0000 UTC }]
May 25 06:16:08.470: INFO: csi-gce-pd-node-h424b bootstrap-e2e-minion-group-nndt Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:44:59 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:08 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:08 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:44:59 +0000 UTC }]
May 25 06:16:08.470: INFO: csi-gce-pd-node-q6qhv bootstrap-e2e-minion-group-kn6c Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:00 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:10 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:45:10 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 05:44:59 +0000 UTC }]
May 25 06:16:08.470: INFO:
May 25 06:16:08.893: INFO: Unable to fetch snapshotting-3720-6758/csi-gce-pd-node-fjwm7/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-fjwm7)
May 25 06:16:08.939: INFO: Unable to fetch snapshotting-3720-6758/csi-gce-pd-node-fjwm7/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-fjwm7)
May 25 06:16:09.077: INFO: Unable to fetch snapshotting-3720-6758/csi-gce-pd-node-q6qhv/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-q6qhv)
May 25 06:16:09.117: INFO: Unable to fetch snapshotting-3720-6758/csi-gce-pd-node-q6qhv/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-q6qhv)
May 25 06:16:09.167: INFO:
Logging node info for node bootstrap-e2e-master
May 25 06:16:09.217: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 6285 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 06:16:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:09.217: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 06:16:09.268: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 06:16:09.335: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.337: INFO: Container etcd-container ready: true, restart count 0
May 25 06:16:09.337: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.337: INFO: Container kube-apiserver ready: true, restart count 0
May 25 06:16:09.337: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.338: INFO: Container kube-scheduler ready: true, restart count 0
May 25 06:16:09.338: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:09.338: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:09.338: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:09.338: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.338: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 06:16:09.338: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.338: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 06:16:09.338: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.338: INFO: Container etcd-container ready: true, restart count 0
May 25 06:16:09.338: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.338: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 06:16:09.338: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.338: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 06:16:09.515: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 06:16:09.515: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 06:16:09.557: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 6283 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 05:50:50 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:11:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:13:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:09.557: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 06:16:09.602: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 06:16:09.648: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.648: INFO: Container coredns ready: true, restart count 0
May 25 06:16:09.648: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.648: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:16:09.648: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:09.648: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:09.648: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:09.648: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.648: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:16:09.817: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 06:16:09.817: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 06:16:09.878: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 6297 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:11:10 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:13:13 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:10 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:10 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:10 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:11:10 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:09.878: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 06:16:09.923: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 06:16:09.995: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.996: INFO: Container coredns ready: true, restart count 0
May 25 06:16:09.996: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.996: INFO: Container autoscaler ready: true, restart count 0
May 25 06:16:09.996: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.996: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 06:16:09.996: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.996: INFO: Container default-http-backend ready: true, restart count 0
May 25 06:16:09.996: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.996: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:16:09.996: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:09.996: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:16:09.996: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:09.996: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:09.996: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:10.191: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 06:16:10.191: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 06:16:10.256: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 6296 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:11:07 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:13:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:07 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:07 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:07 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:11:07 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:10.256: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 06:16:10.327: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 06:16:10.383: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:10.383: INFO: Container metrics-server ready: true, restart count 0
May 25 06:16:10.383: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 06:16:10.383: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:10.383: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:16:10.383: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:10.383: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:10.383: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:10.383: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:10.383: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:16:10.552: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-3720-6758] to vanish
[AfterEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource]
test/e2e/framework/framework.go:187
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-3720".
�[1mSTEP�[0m: Found 0 events.
May 25 06:16:16.717: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 06:16:16.717: INFO:
May 25 06:16:16.759: INFO:
Logging node info for node bootstrap-e2e-master
May 25 06:16:16.800: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 6285 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 06:16:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:16:01 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:16.800: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 06:16:16.843: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 06:16:17.099: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 06:16:17.099: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 06:16:17.099: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container etcd-container ready: true, restart count 0
May 25 06:16:17.099: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 06:16:17.099: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 06:16:17.099: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:17.099: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:17.099: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:17.099: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container etcd-container ready: true, restart count 0
May 25 06:16:17.099: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container kube-apiserver ready: true, restart count 0
May 25 06:16:17.099: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.099: INFO: Container kube-scheduler ready: true, restart count 0
May 25 06:16:17.303: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 06:16:17.303: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 06:16:17.344: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 6283 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 05:50:50 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 06:11:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 06:13:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:11:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:17.345: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 06:16:17.387: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 06:16:17.433: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.433: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:16:17.433: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:17.433: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:17.433: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:17.433: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.433: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:16:17.433: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.433: INFO: Container coredns ready: true, restart count 0
May 25 06:16:17.687: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 06:16:17.687: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 06:16:17.729: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 6332 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 06:13:13 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 06:16:16 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:13:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:16 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:16 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:16 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:16:16 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:17.730: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 06:16:17.773: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 06:16:17.826: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.826: INFO: Container autoscaler ready: true, restart count 0
May 25 06:16:17.826: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.826: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 06:16:17.826: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.826: INFO: Container default-http-backend ready: true, restart count 0
May 25 06:16:17.826: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.826: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:16:17.826: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.826: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:16:17.826: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:17.826: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:17.826: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:17.826: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:17.826: INFO: Container coredns ready: true, restart count 0
May 25 06:16:18.013: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 06:16:18.014: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 06:16:18.056: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 6317 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 06:13:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 06:16:14 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 06:13:15 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 06:16:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 06:16:14 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 06:16:18.056: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 06:16:18.109: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 06:16:18.157: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:18.157: INFO: Container kube-proxy ready: true, restart count 0
May 25 06:16:18.157: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:18.157: INFO: Container metadata-proxy ready: true, restart count 0
May 25 06:16:18.157: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 06:16:18.157: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 06:16:18.157: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 06:16:18.157: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 06:16:18.157: INFO: Container metrics-server ready: true, restart count 0
May 25 06:16:18.157: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 06:16:18.326: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
May 25 06:16:18.326: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
�[1mSTEP�[0m: Destroying namespace "snapshotting-3720" for this suite.
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sDynamic\sSnapshot\s\(retain\spolicy\)\]\ssnapshottable\[Feature\:VolumeSnapshotDataSource\]\svolume\ssnapshot\scontroller\s\sshould\scheck\ssnapshot\sfields\,\scheck\srestore\scorrectly\sworks\safter\smodifying\ssource\sdata\,\scheck\sdeletion\s\(persistent\)$'
test/e2e/storage/testsuites/snapshottable.go:278
May 25 06:51:44.150: Unexpected error:
<*errors.errorString | 0xc0010baba0>: {
s: "VolumeSnapshot snapshot-dg5l9 is not ready within 5m0s",
}
VolumeSnapshot snapshot-dg5l9 is not ready within 5m0s
occurred
test/e2e/storage/utils/snapshot.go:86
[BeforeEach] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource]
test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource]
test/e2e/framework/framework.go:186
�[1mSTEP�[0m: Creating a kubernetes client
May 25 06:46:16.603: INFO: >>> kubeConfig: /workspace/.kube/config
�[1mSTEP�[0m: Building a namespace api object, basename snapshotting
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
�[1mSTEP�[0m: Waiting for kube-root-ca.crt to be provisioned in namespace
[It] should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
test/e2e/storage/testsuites/snapshottable.go:278
�[1mSTEP�[0m: deploying csi gce-pd driver
�[1mSTEP�[0m: Building a driver namespace object, basename snapshotting-1544
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
May 25 06:46:17.237: INFO: Found CI service account key at /etc/service-account/service-account.json
May 25 06:46:17.237: INFO: Running cp [/etc/service-account/service-account.json /tmp/3fa009ad-03ff-4c8f-84f0-9aecbe978b99/cloud-sa.json]
May 25 06:46:17.286: INFO: Shredding file /tmp/3fa009ad-03ff-4c8f-84f0-9aecbe978b99/cloud-sa.json
May 25 06:46:17.286: INFO: Running shred [--remove /tmp/3fa009ad-03ff-4c8f-84f0-9aecbe978b99/cloud-sa.json]
May 25 06:46:17.321: INFO: File /tmp/3fa009ad-03ff-4c8f-84f0-9aecbe978b99/cloud-sa.json successfully shredded
May 25 06:46:17.329: INFO: creating *v1.ServiceAccount: snapshotting-1544-2274/csi-attacher
May 25 06:46:17.371: INFO: creating *v1.ClusterRole: external-attacher-runner-snapshotting-1544
May 25 06:46:17.371: INFO: Define cluster role external-attacher-runner-snapshotting-1544
May 25 06:46:17.416: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-1544
May 25 06:46:17.459: INFO: creating *v1.Role: snapshotting-1544-2274/external-attacher-cfg-snapshotting-1544
May 25 06:46:17.506: INFO: creating *v1.RoleBinding: snapshotting-1544-2274/csi-attacher-role-cfg
May 25 06:46:17.549: INFO: creating *v1.ServiceAccount: snapshotting-1544-2274/csi-provisioner
May 25 06:46:17.592: INFO: creating *v1.ClusterRole: external-provisioner-runner-snapshotting-1544
May 25 06:46:17.593: INFO: Define cluster role external-provisioner-runner-snapshotting-1544
May 25 06:46:17.637: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-1544
May 25 06:46:17.680: INFO: creating *v1.Role: snapshotting-1544-2274/external-provisioner-cfg-snapshotting-1544
May 25 06:46:17.723: INFO: creating *v1.RoleBinding: snapshotting-1544-2274/csi-provisioner-role-cfg
May 25 06:46:17.767: INFO: creating *v1.ServiceAccount: snapshotting-1544-2274/csi-gce-pd-controller-sa
May 25 06:46:17.812: INFO: creating *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-1544
May 25 06:46:17.812: INFO: Define cluster role csi-gce-pd-provisioner-role-snapshotting-1544
May 25 06:46:17.858: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-1544
May 25 06:46:17.901: INFO: creating *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-1544
May 25 06:46:17.901: INFO: Define cluster role csi-gce-pd-attacher-role-snapshotting-1544
May 25 06:46:17.943: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-1544
May 25 06:46:17.986: INFO: creating *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-1544
May 25 06:46:17.986: INFO: Define cluster role csi-gce-pd-resizer-role-snapshotting-1544
May 25 06:46:18.028: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-1544
May 25 06:46:18.071: INFO: creating *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-1544
May 25 06:46:18.071: INFO: Define cluster role csi-gce-pd-snapshotter-role-snapshotting-1544
May 25 06:46:18.113: INFO: creating *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-1544
May 25 06:46:18.155: INFO: creating *v1.Role: snapshotting-1544-2274/csi-gce-pd-leaderelection-role-snapshotting-1544
May 25 06:46:18.198: INFO: creating *v1.RoleBinding: snapshotting-1544-2274/csi-gce-pd-controller-leaderelection-binding
May 25 06:46:18.243: INFO: creating *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-1544
May 25 06:46:18.320: INFO: creating *v1.DaemonSet: snapshotting-1544-2274/csi-gce-pd-node
May 25 06:46:18.366: INFO: creating *v1.StatefulSet: snapshotting-1544-2274/csi-gce-pd-controller
May 25 06:46:18.565: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-gw73
May 25 06:46:20.677: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-kn6c
May 25 06:46:22.773: INFO: waiting for CSIDriver pd.csi.storage.gke.io to register on node bootstrap-e2e-minion-group-nndt
May 25 06:46:22.824: INFO: Creating resource for dynamic PV
May 25 06:46:22.824: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(pd.csi.storage.gke.io) supported size:{ 1Mi}
�[1mSTEP�[0m: creating a StorageClass snapshotting-1544556c6
�[1mSTEP�[0m: creating a claim
May 25 06:46:22.903: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
�[1mSTEP�[0m: [init] starting a pod to use the claim
�[1mSTEP�[0m: [init] check pod success
May 25 06:46:23.154: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-xgjdr" in namespace "snapshotting-1544" to be "Succeeded or Failed"
May 25 06:46:23.217: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 62.647566ms
May 25 06:46:25.259: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.1052955s
May 25 06:46:27.259: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.104869374s
May 25 06:46:29.259: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.105157251s
May 25 06:46:31.258: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.104407861s
May 25 06:46:33.260: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.105801331s
May 25 06:46:35.260: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 12.105775761s
May 25 06:46:37.258: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 14.104200823s
May 25 06:46:39.259: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Pending", Reason="", readiness=false. Elapsed: 16.105036364s
May 25 06:46:41.259: INFO: Pod "pvc-snapshottable-tester-xgjdr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.104832857s
�[1mSTEP�[0m: Saw pod success
May 25 06:46:41.259: INFO: Pod "pvc-snapshottable-tester-xgjdr" satisfied condition "Succeeded or Failed"
�[1mSTEP�[0m: [init] checking the claim
May 25 06:46:41.300: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pd.csi.storage.gke.ionhbxr] to have phase Bound
May 25 06:46:41.341: INFO: PersistentVolumeClaim pd.csi.storage.gke.ionhbxr found and phase=Bound (41.270797ms)
�[1mSTEP�[0m: [init] checking the PV
�[1mSTEP�[0m: [init] deleting the pod
May 25 06:46:41.481: INFO: Pod pvc-snapshottable-tester-xgjdr has the following logs:
May 25 06:46:41.481: INFO: Deleting pod "pvc-snapshottable-tester-xgjdr" in namespace "snapshotting-1544"
May 25 06:46:41.536: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-xgjdr" to be fully deleted
�[1mSTEP�[0m: [init] waiting until the node=bootstrap-e2e-minion-group-gw73 is not using the volume=projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/disks/pvc-32744921-51d9-4f41-97dd-edd18c268f43
May 25 06:46:41.619: INFO: current volumes in use: [kubernetes.io/csi/pd.csi.storage.gke.io^projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/disks/pvc-32744921-51d9-4f41-97dd-edd18c268f43]
May 25 06:46:43.663: INFO: current volumes in use: []
May 25 06:46:43.663: INFO: WaitUntil finished successfully after 2.085497945s
�[1mSTEP�[0m: creating a SnapshotClass
�[1mSTEP�[0m: creating a dynamic VolumeSnapshot
May 25 06:46:43.792: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-dg5l9 to become ready
May 25 06:46:43.836: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:46:45.880: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:46:47.935: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:46:49.979: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:46:52.022: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:46:54.064: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:46:56.106: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:46:58.148: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:00.195: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:02.237: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:04.279: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:06.322: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:08.364: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:10.407: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:12.450: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:14.492: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:16.535: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:18.578: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:20.622: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:22.665: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:24.707: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:26.750: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:28.793: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:30.836: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:32.880: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:34.922: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:36.966: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:39.009: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:41.052: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:43.094: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:45.141: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:47.183: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:49.226: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:51.271: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:53.313: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:55.359: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:57.402: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:47:59.445: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:01.488: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:03.532: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:05.576: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:07.618: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:09.661: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:11.704: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:13.746: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:15.789: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:17.831: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:19.874: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:21.916: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:23.959: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:26.001: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:28.043: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:30.087: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:32.130: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:34.173: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:36.215: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:38.264: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:40.308: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:42.351: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:44.393: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:46.437: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:48.481: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:50.523: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:52.569: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:54.613: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:56.655: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:48:58.698: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:00.741: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:02.784: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:04.827: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:06.868: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:08.911: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:10.954: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:12.997: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:15.039: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:17.083: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:19.124: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:21.166: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:23.209: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:25.251: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:27.293: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:29.336: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:31.378: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:33.420: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:35.464: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:37.507: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:39.549: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:41.591: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:43.634: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:45.676: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:47.723: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:49.765: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:51.807: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:53.875: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:55.916: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:49:57.960: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:00.003: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:02.045: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:04.087: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:06.131: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:08.173: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:10.216: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:12.259: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:14.301: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:16.345: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:18.389: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:20.431: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:22.473: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:24.515: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:26.562: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:28.605: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:30.647: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:32.690: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:34.732: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:36.775: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:38.818: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:40.866: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:42.907: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:44.951: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:46.995: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:49.038: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:51.080: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:53.123: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:55.166: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:57.208: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:50:59.251: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:01.295: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:03.338: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:05.381: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:07.423: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:09.467: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:11.509: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:13.551: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:15.594: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:17.638: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:19.681: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:21.723: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:23.765: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:25.808: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:27.853: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:29.895: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:31.937: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:33.979: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:36.022: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:38.064: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:40.107: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:42.149: INFO: VolumeSnapshot snapshot-dg5l9 found but is not ready.
May 25 06:51:44.149: INFO: WaitUntil failed after reaching the timeout 5m0s
May 25 06:51:44.150: FAIL: Unexpected error:
<*errors.errorString | 0xc0010baba0>: {
s: "VolumeSnapshot snapshot-dg5l9 is not ready within 5m0s",
}
VolumeSnapshot snapshot-dg5l9 is not ready within 5m0s
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.GetSnapshotContentFromSnapshot({0x79eab80, 0xc0047edba0?}, 0xc000e12d70)
test/e2e/storage/utils/snapshot.go:86 +0x1ad
k8s.io/kubernetes/test/e2e/storage/framework.CreateSnapshotResource({0x7f220008b9b0, 0xc000569ee0}, 0xc0040a0360, {{0x71ba8d5, 0x20}, {0x0, 0x0}, {0x7133ab2, 0x9}, {0x0, ...}, ...}, ...)
test/e2e/storage/framework/snapshot_resource.go:92 +0x246
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2()
test/e2e/storage/testsuites/snapshottable.go:359 +0xb3d
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
�[1mSTEP�[0m: checking the snapshot
�[1mSTEP�[0m: checking the SnapshotContent
�[1mSTEP�[0m: Modifying source data test
�[1mSTEP�[0m: modifying the data in the source PVC
May 25 06:51:44.283: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-qb2xk" in namespace "snapshotting-1544" to be "Succeeded or Failed"
May 25 06:51:44.324: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Pending", Reason="", readiness=false. Elapsed: 40.772438ms
May 25 06:51:46.366: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.082852091s
May 25 06:51:48.368: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.0841826s
May 25 06:51:50.366: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Pending", Reason="", readiness=false. Elapsed: 6.082334014s
May 25 06:51:52.368: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Pending", Reason="", readiness=false. Elapsed: 8.08425981s
May 25 06:51:54.366: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Pending", Reason="", readiness=false. Elapsed: 10.083112104s
May 25 06:51:56.366: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Running", Reason="", readiness=false. Elapsed: 12.082477846s
May 25 06:51:58.406: INFO: Pod "pvc-snapshottable-data-tester-qb2xk": Phase="Succeeded", Reason="", readiness=false. Elapsed: 14.122332071s
�[1mSTEP�[0m: Saw pod success
May 25 06:51:58.406: INFO: Pod "pvc-snapshottable-data-tester-qb2xk" satisfied condition "Succeeded or Failed"
May 25 06:51:58.537: INFO: Pod pvc-snapshottable-data-tester-qb2xk has the following logs:
May 25 06:51:58.537: INFO: Deleting pod "pvc-snapshottable-data-tester-qb2xk" in namespace "snapshotting-1544"
May 25 06:51:58.589: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-qb2xk" to be fully deleted
�[1mSTEP�[0m: creating a pvc from the snapshot
�[1mSTEP�[0m: starting a pod to use the snapshot
May 25 06:51:58.717: INFO: Waiting up to 15m0s for pod "restored-pvc-tester-gb99j" in namespace "snapshotting-1544" to be "running"
May 25 06:51:58.758: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 40.571505ms
May 25 06:52:00.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2.082505566s
May 25 06:52:02.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4.083237012s
May 25 06:52:04.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6.081895248s
May 25 06:52:06.804: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8.086364523s
May 25 06:52:08.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10.082628892s
May 25 06:52:10.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12.083578816s
May 25 06:52:12.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14.083545976s
May 25 06:52:14.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 16.082491884s
May 25 06:52:16.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 18.082268843s
May 25 06:52:18.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 20.083545855s
May 25 06:52:20.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 22.082878458s
May 25 06:52:22.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 24.084605674s
May 25 06:52:24.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 26.082586029s
May 25 06:52:26.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 28.083441227s
May 25 06:52:28.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 30.08272804s
May 25 06:52:30.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 32.082173646s
May 25 06:52:32.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 34.084678735s
May 25 06:52:34.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 36.082494132s
May 25 06:52:36.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 38.082850543s
May 25 06:52:38.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 40.083065379s
May 25 06:52:40.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 42.083114882s
May 25 06:52:42.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 44.085043546s
May 25 06:52:44.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 46.08306287s
May 25 06:52:46.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 48.084034703s
May 25 06:52:48.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 50.083027106s
May 25 06:52:50.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 52.082264237s
May 25 06:52:52.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 54.083768333s
May 25 06:52:54.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 56.08338227s
May 25 06:52:56.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 58.084219306s
May 25 06:52:58.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.082962744s
May 25 06:53:00.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.082780321s
May 25 06:53:02.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.082083847s
May 25 06:53:04.803: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.085183122s
May 25 06:53:06.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.083338675s
May 25 06:53:08.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.084840482s
May 25 06:53:10.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.082074558s
May 25 06:53:12.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.082074338s
May 25 06:53:14.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.082346994s
May 25 06:53:16.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.082011494s
May 25 06:53:18.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.082912228s
May 25 06:53:20.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.083891789s
May 25 06:53:22.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.08383551s
May 25 06:53:24.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.114475475s
May 25 06:53:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.114499729s
May 25 06:53:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.114350521s
May 25 06:53:30.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.114568194s
May 25 06:53:32.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.114705102s
May 25 06:53:34.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.113912574s
May 25 06:53:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.114282119s
May 25 06:53:38.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.118290913s
May 25 06:53:40.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.082991762s
May 25 06:53:42.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.115650527s
May 25 06:53:44.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.11401002s
May 25 06:53:46.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.117624006s
May 25 06:53:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.11500407s
May 25 06:53:50.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.114421503s
May 25 06:53:52.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.083806096s
May 25 06:53:54.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.082546604s
May 25 06:53:56.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.114599152s
May 25 06:53:58.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.113926016s
May 25 06:54:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.114353049s
May 25 06:54:02.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.114995736s
May 25 06:54:04.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.114966997s
May 25 06:54:06.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.115290539s
May 25 06:54:08.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.113929382s
May 25 06:54:10.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.114646027s
May 25 06:54:12.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.114297436s
May 25 06:54:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.114800424s
May 25 06:54:16.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.114347712s
May 25 06:54:18.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.114479425s
May 25 06:54:20.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.08216476s
May 25 06:54:22.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.114390603s
May 25 06:54:24.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.113801552s
May 25 06:54:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.114124913s
May 25 06:54:28.844: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.126252369s
May 25 06:54:30.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.082607659s
May 25 06:54:32.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.114867604s
May 25 06:54:34.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.1148733s
May 25 06:54:36.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.114064355s
May 25 06:54:38.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.114216322s
May 25 06:54:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.114447585s
May 25 06:54:42.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.115108204s
May 25 06:54:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.114118941s
May 25 06:54:46.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.118903736s
May 25 06:54:48.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.113742482s
May 25 06:54:50.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.113654238s
May 25 06:54:52.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.082303641s
May 25 06:54:54.821: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.103190993s
May 25 06:54:56.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.113984907s
May 25 06:54:58.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.116211428s
May 25 06:55:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.115053386s
May 25 06:55:02.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.084237232s
May 25 06:55:04.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.11759825s
May 25 06:55:06.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.116031483s
May 25 06:55:08.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.118337932s
May 25 06:55:10.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.116975496s
May 25 06:55:12.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.11699162s
May 25 06:55:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.114362415s
May 25 06:55:16.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.114541951s
May 25 06:55:18.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.119345337s
May 25 06:55:20.874: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.157010481s
May 25 06:55:22.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.115588795s
May 25 06:55:24.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.113687173s
May 25 06:55:26.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.115108518s
May 25 06:55:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.114345812s
May 25 06:55:30.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.118904453s
May 25 06:55:32.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.114399045s
May 25 06:55:34.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.115953756s
May 25 06:55:36.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.083462472s
May 25 06:55:38.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.115224166s
May 25 06:55:40.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.119292232s
May 25 06:55:42.841: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.123326697s
May 25 06:55:44.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.082744021s
May 25 06:55:46.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.119203531s
May 25 06:55:48.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.119207293s
May 25 06:55:50.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.117234885s
May 25 06:55:52.805: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.087168602s
May 25 06:55:54.805: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.087229101s
May 25 06:55:56.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.119193366s
May 25 06:55:58.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.084576169s
May 25 06:56:00.827: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.109374085s
May 25 06:56:02.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.083987927s
May 25 06:56:04.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.119027995s
May 25 06:56:06.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.119211587s
May 25 06:56:08.809: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.091287703s
May 25 06:56:10.805: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.087267068s
May 25 06:56:12.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.117856216s
May 25 06:56:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.114284271s
May 25 06:56:16.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.114428209s
May 25 06:56:18.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.083248698s
May 25 06:56:20.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.116549148s
May 25 06:56:22.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.118546011s
May 25 06:56:24.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.114704088s
May 25 06:56:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.114273114s
May 25 06:56:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.114376711s
May 25 06:56:30.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.114306982s
May 25 06:56:32.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.115111627s
May 25 06:56:34.807: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.089521323s
May 25 06:56:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.114607321s
May 25 06:56:38.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.115150238s
May 25 06:56:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.11432756s
May 25 06:56:42.846: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.128695158s
May 25 06:56:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.114990385s
May 25 06:56:46.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.114887499s
May 25 06:56:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.114830568s
May 25 06:56:50.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.116602034s
May 25 06:56:52.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.082683876s
May 25 06:56:54.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.114557082s
May 25 06:56:56.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.114532835s
May 25 06:56:58.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m0.082203996s
May 25 06:57:00.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m2.11525402s
May 25 06:57:02.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m4.11432114s
May 25 06:57:04.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m6.115093978s
May 25 06:57:06.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m8.114486206s
May 25 06:57:08.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m10.082656291s
May 25 06:57:10.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m12.1143812s
May 25 06:57:12.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m14.116177279s
May 25 06:57:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m16.114776511s
May 25 06:57:16.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m18.115424676s
May 25 06:57:18.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m20.114397777s
May 25 06:57:20.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m22.117419429s
May 25 06:57:22.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m24.114548987s
May 25 06:57:24.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m26.116032778s
May 25 06:57:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m28.11470891s
May 25 06:57:28.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m30.114079401s
May 25 06:57:30.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m32.113850651s
May 25 06:57:32.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m34.114603335s
May 25 06:57:34.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m36.114018559s
May 25 06:57:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m38.114345818s
May 25 06:57:38.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m40.114275514s
May 25 06:57:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m42.114453789s
May 25 06:57:42.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m44.114611047s
May 25 06:57:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m46.114220564s
May 25 06:57:46.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m48.114354227s
May 25 06:57:48.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m50.11403783s
May 25 06:57:50.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m52.114478103s
May 25 06:57:52.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m54.083465674s
May 25 06:57:54.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m56.113656797s
May 25 06:57:56.804: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 5m58.08677137s
May 25 06:57:58.863: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m0.145491075s
May 25 06:58:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m2.114909018s
May 25 06:58:02.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m4.115394006s
May 25 06:58:04.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m6.114696343s
May 25 06:58:06.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m8.081976605s
May 25 06:58:08.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m10.114695296s
May 25 06:58:10.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m12.11447094s
May 25 06:58:12.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m14.115288716s
May 25 06:58:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m16.114428579s
May 25 06:58:16.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m18.113895636s
May 25 06:58:18.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m20.114173263s
May 25 06:58:20.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m22.114598155s
May 25 06:58:22.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m24.115418888s
May 25 06:58:24.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m26.114050065s
May 25 06:58:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m28.114582077s
May 25 06:58:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m30.114764863s
May 25 06:58:30.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m32.114239335s
May 25 06:58:32.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m34.115890811s
May 25 06:58:34.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m36.114060895s
May 25 06:58:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m38.114331801s
May 25 06:58:38.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m40.114507718s
May 25 06:58:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m42.11461032s
May 25 06:58:42.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m44.115532047s
May 25 06:58:44.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m46.115191998s
May 25 06:58:46.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m48.115124966s
May 25 06:58:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m50.114549024s
May 25 06:58:50.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m52.082950457s
May 25 06:58:52.802: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m54.084983632s
May 25 06:58:54.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m56.1140025s
May 25 06:58:56.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 6m58.114965496s
May 25 06:58:58.911: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m0.194056561s
May 25 06:59:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m2.114194208s
May 25 06:59:02.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m4.114274959s
May 25 06:59:04.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m6.116511441s
May 25 06:59:06.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m8.114220679s
May 25 06:59:08.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m10.114465912s
May 25 06:59:10.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m12.113691378s
May 25 06:59:12.815: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m14.097209256s
May 25 06:59:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m16.114135713s
May 25 06:59:16.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m18.113745866s
May 25 06:59:18.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m20.11434777s
May 25 06:59:20.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m22.114691909s
May 25 06:59:22.866: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m24.148353328s
May 25 06:59:24.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m26.11432249s
May 25 06:59:26.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m28.115416028s
May 25 06:59:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m30.114262328s
May 25 06:59:30.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m32.082666573s
May 25 06:59:32.803: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m34.085635637s
May 25 06:59:34.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m36.114868427s
May 25 06:59:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m38.114609001s
May 25 06:59:38.840: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m40.122485534s
May 25 06:59:40.871: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m42.153749233s
May 25 06:59:42.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m44.117532907s
May 25 06:59:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m46.11434201s
May 25 06:59:46.841: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m48.123110659s
May 25 06:59:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m50.114326778s
May 25 06:59:50.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m52.114011492s
May 25 06:59:52.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m54.082655171s
May 25 06:59:54.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m56.082536649s
May 25 06:59:56.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 7m58.114784243s
May 25 06:59:58.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m0.114656384s
May 25 07:00:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m2.114504943s
May 25 07:00:02.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m4.082608491s
May 25 07:00:04.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m6.114040299s
May 25 07:00:06.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m8.08289167s
May 25 07:00:08.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m10.114388149s
May 25 07:00:10.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m12.11490684s
May 25 07:00:12.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m14.114968311s
May 25 07:00:14.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m16.11913518s
May 25 07:00:16.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m18.119160102s
May 25 07:00:18.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m20.119212515s
May 25 07:00:20.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m22.11922565s
May 25 07:00:22.805: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m24.087191276s
May 25 07:00:24.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m26.119170885s
May 25 07:00:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m28.114652949s
May 25 07:00:28.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m30.113975405s
May 25 07:00:30.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m32.115158227s
May 25 07:00:32.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m34.115183831s
May 25 07:00:34.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m36.114631279s
May 25 07:00:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m38.114623735s
May 25 07:00:38.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m40.114653693s
May 25 07:00:40.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m42.082935943s
May 25 07:00:42.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m44.11675309s
May 25 07:00:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m46.114379722s
May 25 07:00:46.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m48.113888659s
May 25 07:00:48.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m50.119447249s
May 25 07:00:50.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m52.114658983s
May 25 07:00:52.819: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m54.101880037s
May 25 07:00:54.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m56.114015235s
May 25 07:00:56.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 8m58.116627814s
May 25 07:00:58.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m0.114723244s
May 25 07:01:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m2.114108059s
May 25 07:01:02.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m4.11490179s
May 25 07:01:04.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m6.114884609s
May 25 07:01:06.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m8.114448699s
May 25 07:01:08.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m10.114289462s
May 25 07:01:10.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m12.114295272s
May 25 07:01:12.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m14.115399003s
May 25 07:01:14.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m16.115110831s
May 25 07:01:16.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m18.115082241s
May 25 07:01:18.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m20.114739664s
May 25 07:01:20.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m22.115092242s
May 25 07:01:22.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m24.083876859s
May 25 07:01:24.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m26.114302815s
May 25 07:01:26.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m28.081361106s
May 25 07:01:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m30.114260617s
May 25 07:01:30.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m32.114207099s
May 25 07:01:32.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m34.114607863s
May 25 07:01:34.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m36.116002151s
May 25 07:01:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m38.114396616s
May 25 07:01:38.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m40.117403577s
May 25 07:01:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m42.114344603s
May 25 07:01:42.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m44.083575168s
May 25 07:01:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m46.114925352s
May 25 07:01:46.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m48.114489145s
May 25 07:01:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m50.114407353s
May 25 07:01:50.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m52.11417004s
May 25 07:01:52.804: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m54.086310274s
May 25 07:01:54.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m56.114770731s
May 25 07:01:56.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 9m58.115117826s
May 25 07:01:58.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m0.114898723s
May 25 07:02:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m2.114359684s
May 25 07:02:02.867: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m4.149928147s
May 25 07:02:04.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m6.11414655s
May 25 07:02:06.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m8.114693949s
May 25 07:02:08.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m10.113936972s
May 25 07:02:10.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m12.11396231s
May 25 07:02:12.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m14.114626367s
May 25 07:02:14.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m16.113972713s
May 25 07:02:16.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m18.08262269s
May 25 07:02:18.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m20.113726687s
May 25 07:02:20.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m22.115697225s
May 25 07:02:22.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m24.114581903s
May 25 07:02:24.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m26.113993062s
May 25 07:02:26.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m28.113639337s
May 25 07:02:28.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m30.116497378s
May 25 07:02:30.841: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m32.123626303s
May 25 07:02:32.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m34.119082861s
May 25 07:02:34.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m36.114191702s
May 25 07:02:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m38.114275849s
May 25 07:02:38.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m40.113645598s
May 25 07:02:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m42.114472088s
May 25 07:02:42.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m44.115718666s
May 25 07:02:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m46.114352524s
May 25 07:02:46.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m48.082532008s
May 25 07:02:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m50.114424006s
May 25 07:02:50.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m52.114600927s
May 25 07:02:52.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m54.083780862s
May 25 07:02:54.919: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m56.201711539s
May 25 07:02:56.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 10m58.113942548s
May 25 07:02:58.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m0.114022149s
May 25 07:03:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m2.114154241s
May 25 07:03:02.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m4.114948471s
May 25 07:03:04.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m6.114390781s
May 25 07:03:06.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m8.114049551s
May 25 07:03:08.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m10.114404692s
May 25 07:03:10.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m12.114756974s
May 25 07:03:12.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m14.11436955s
May 25 07:03:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m16.114412363s
May 25 07:03:16.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m18.114630443s
May 25 07:03:18.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m20.114294899s
May 25 07:03:20.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m22.114876374s
May 25 07:03:22.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m24.117472662s
May 25 07:03:24.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m26.114016783s
May 25 07:03:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m28.114548315s
May 25 07:03:28.799: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m30.081741233s
May 25 07:03:30.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m32.113493693s
May 25 07:03:32.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m34.118623039s
May 25 07:03:34.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m36.114614054s
May 25 07:03:36.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m38.116779962s
May 25 07:03:38.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m40.114910268s
May 25 07:03:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m42.114131903s
May 25 07:03:42.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m44.115128446s
May 25 07:03:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m46.114789292s
May 25 07:03:46.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m48.114454449s
May 25 07:03:48.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m50.119176793s
May 25 07:03:50.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m52.119207917s
May 25 07:03:52.809: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m54.09117547s
May 25 07:03:54.805: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m56.08732533s
May 25 07:03:56.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 11m58.119169806s
May 25 07:03:58.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m0.115202274s
May 25 07:04:00.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m2.082176609s
May 25 07:04:02.805: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m4.087248899s
May 25 07:04:04.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m6.113924842s
May 25 07:04:06.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m8.113843145s
May 25 07:04:08.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m10.113763377s
May 25 07:04:10.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m12.114349402s
May 25 07:04:12.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m14.11486546s
May 25 07:04:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m16.114212326s
May 25 07:04:16.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m18.114066467s
May 25 07:04:18.800: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m20.082565509s
May 25 07:04:20.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m22.114389673s
May 25 07:04:22.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m24.115416905s
May 25 07:04:24.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m26.114346105s
May 25 07:04:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m28.11417159s
May 25 07:04:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m30.114336763s
May 25 07:04:30.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m32.114145522s
May 25 07:04:32.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m34.114392424s
May 25 07:04:34.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m36.114502881s
May 25 07:04:36.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m38.113860757s
May 25 07:04:38.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m40.113929614s
May 25 07:04:40.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m42.114378781s
May 25 07:04:42.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m44.1150381s
May 25 07:04:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m46.114257697s
May 25 07:04:46.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m48.114936883s
May 25 07:04:48.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m50.115332191s
May 25 07:04:50.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m52.114184546s
May 25 07:04:52.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m54.115869847s
May 25 07:04:54.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m56.11426073s
May 25 07:04:56.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 12m58.114078773s
May 25 07:04:58.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m0.114694669s
May 25 07:05:00.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m2.113216438s
May 25 07:05:02.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m4.115714864s
May 25 07:05:04.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m6.114082588s
May 25 07:05:06.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m8.113897126s
May 25 07:05:08.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m10.119628132s
May 25 07:05:10.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m12.083108387s
May 25 07:05:12.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m14.114383182s
May 25 07:05:14.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m16.114454876s
May 25 07:05:16.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m18.11395442s
May 25 07:05:18.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m20.119152158s
May 25 07:05:20.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m22.115088056s
May 25 07:05:22.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m24.11535056s
May 25 07:05:24.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m26.114077185s
May 25 07:05:26.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m28.113949773s
May 25 07:05:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m30.114120819s
May 25 07:05:30.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m32.114420242s
May 25 07:05:32.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m34.114008037s
May 25 07:05:34.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m36.113783425s
May 25 07:05:36.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m38.115222136s
May 25 07:05:38.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m40.114751957s
May 25 07:05:40.853: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m42.135154593s
May 25 07:05:42.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m44.11460972s
May 25 07:05:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m46.114109991s
May 25 07:05:46.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m48.114473513s
May 25 07:05:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m50.114612957s
May 25 07:05:50.835: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m52.117848444s
May 25 07:05:52.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m54.114439354s
May 25 07:05:54.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m56.115051395s
May 25 07:05:56.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 13m58.11376393s
May 25 07:05:58.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m0.114356505s
May 25 07:06:00.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m2.114789162s
May 25 07:06:02.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m4.115496942s
May 25 07:06:04.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m6.114699113s
May 25 07:06:06.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m8.113929974s
May 25 07:06:08.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m10.11401246s
May 25 07:06:10.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m12.113577546s
May 25 07:06:12.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m14.115537959s
May 25 07:06:14.836: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m16.118775196s
May 25 07:06:16.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m18.114288297s
May 25 07:06:18.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m20.114713128s
May 25 07:06:20.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m22.11345665s
May 25 07:06:22.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m24.115527955s
May 25 07:06:24.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m26.114669515s
May 25 07:06:26.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m28.114287071s
May 25 07:06:28.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m30.11465269s
May 25 07:06:30.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m32.114995332s
May 25 07:06:32.834: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m34.116716684s
May 25 07:06:34.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m36.115125832s
May 25 07:06:36.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m38.114848132s
May 25 07:06:38.831: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m40.113328449s
May 25 07:06:40.837: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m42.119337393s
May 25 07:06:42.833: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m44.115939707s
May 25 07:06:44.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m46.114736373s
May 25 07:06:46.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m48.114193882s
May 25 07:06:48.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.114209649s
May 25 07:06:50.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.114391055s
May 25 07:06:52.801: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.08320926s
May 25 07:06:54.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.11437114s
May 25 07:06:56.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.114614499s
May 25 07:06:58.832: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.114796272s
May 25 07:06:58.932: INFO: Pod "restored-pvc-tester-gb99j": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.214734428s
May 25 07:06:58.933: INFO: Timed out while waiting for pod snapshotting-1544/restored-pvc-tester-gb99j to be running. Last observed as: <*v1.Pod | 0xc0030f0c00>: {
TypeMeta: {Kind: "", APIVersion: ""},
ObjectMeta: {
Name: "restored-pvc-tester-gb99j",
GenerateName: "restored-pvc-tester-",
Namespace: "snapshotting-1544",
SelfLink: "",
UID: "31c77bca-97d2-4b0c-a85b-67d55b20fa7b",
ResourceVersion: "16583",
Generation: 0,
CreationTimestamp: {
Time: 2022-05-25T06:51:58Z,
},
DeletionTimestamp: nil,
DeletionGracePeriodSeconds: nil,
Labels: {
"app": "restored-pvc-tester",
},
Annotations: nil,
OwnerReferences: nil,
Finalizers: nil,
ManagedFields: [
{
Manager: "e2e.test",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: 2022-05-25T06:51:58Z,
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:metadata\":{\"f:generateName\":{},\"f:labels\":{\".\":{},\"f:app\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"volume-tester\\\"}\":{\".\":{},\"f:command\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:name\":{},\"f:resources\":{},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/mnt/test\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{}}}}},\"f:dnsPolicy\":{},\"f:enableServiceLinks\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:terminationGracePeriodSeconds\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"my-volume\\\"}\":{\".\":{},\"f:name\":{},\"f:persistentVolumeClaim\":{\".\":{},\"f:claimName\":{}}}}}}",
},
Subresource: "",
},
{
Manager: "kube-scheduler",
Operation: "Update",
APIVersion: "v1",
Time: {
Time: 2022-05-25T07:01:58Z,
},
FieldsType: "FieldsV1",
FieldsV1: {
Raw: "{\"f:status\":{\"f:conditions\":{\".\":{},\"k:{\\\"type\\\":\\\"PodScheduled\\\"}\":{\".\":{},\"f:lastProbeTime\":{},\"f:lastTransitionTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}}}}",
},
Subresource: "status",
},
],
},
Spec: {
Volumes: [
{
Name: "my-volume",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: {
ClaimName: "pvc-xmpc5",
ReadOnly: false,
},
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: nil,
PortworxVolume: nil,
ScaleIO: nil,
StorageOS: nil,
CSI: nil,
Ephemeral: nil,
},
},
{
Name: "kube-api-access-7nczs",
VolumeSource: {
HostPath: nil,
EmptyDir: nil,
GCEPersistentDisk: nil,
AWSElasticBlockStore: nil,
GitRepo: nil,
Secret: nil,
NFS: nil,
ISCSI: nil,
Glusterfs: nil,
PersistentVolumeClaim: nil,
RBD: nil,
FlexVolume: nil,
Cinder: nil,
CephFS: nil,
Flocker: nil,
DownwardAPI: nil,
FC: nil,
AzureFile: nil,
ConfigMap: nil,
VsphereVolume: nil,
Quobyte: nil,
AzureDisk: nil,
PhotonPersistentDisk: nil,
Projected: {
Sources: [
{
Secret: nil,
DownwardAPI: nil,
ConfigMap: nil,
ServiceAccountToken: {
Audience: "",
ExpirationSeconds: 3607,
Path: "token",
},
},
{
Secret: nil,
DownwardAPI: nil,
ConfigMap: {
LocalObjectReference: {Name: ...},
Items: [...],
Optional: nil,
},
ServiceAccountToken: nil,
},
{
Secret: nil,
DownwardAPI: {Items: [...]},
ConfigMap: nil,
ServiceAccountToken: nil,
},
],
DefaultMode: 420,
},
PortworxVolume: nil,
ScaleIO: nil,
StorageOS: nil,
CSI: nil,
Ephemeral: nil,
},
},
],
InitContainers: nil,
Containers: [
{
Name: "volume-tester",
Image: "k8s.gcr.io/e2e-test-images/busybox:1.29-2",
Command: ["/bin/sh", "-c", "sleep 300"],
Args: nil,
WorkingDir: "",
Ports: nil,
EnvFrom: nil,
Env: nil,
Resources: {Limits: nil, Requests: nil},
VolumeMounts: [
{
Name: "my-volume",
ReadOnly: false,
MountPath: "/mnt/test",
SubPath: "",
MountPropagation: nil,
SubPathExpr: "",
},
{
Name: "kube-api-access-7nczs",
ReadOnly: true,
MountPath: "/var/run/secrets/kubernetes.io/serviceaccount",
SubPath: "",
MountPropagation: nil,
SubPathExpr: "",
},
],
VolumeDevices: nil,
LivenessProbe: nil,
ReadinessProbe: nil,
StartupProbe: nil,
Lifecycle: nil,
TerminationMessagePath: "/dev/termination-log",
TerminationMessagePolicy: "File",
ImagePullPolicy: "IfNotPresent",
SecurityContext: nil,
Stdin: false,
StdinOnce: false,
TTY: false,
},
],
EphemeralContainers: nil,
RestartPolicy: "Never",
TerminationGracePeriodSeconds: 30,
ActiveDeadlineSeconds: nil,
DNSPolicy: "ClusterFirst",
NodeSelector: nil,
ServiceAccountName: "default",
DeprecatedServiceAccount: "default",
AutomountServiceAccountToken: nil,
NodeName: "",
HostNetwork: false,
HostPID: false,
HostIPC: false,
ShareProcessNamespace: nil,
SecurityContext: {
SELinuxOptions: nil,
WindowsOptions: nil,
RunAsUser: nil,
RunAsGroup: nil,
RunAsNonRoot: nil,
SupplementalGroups: nil,
FSGroup: nil,
Sysctls: nil,
FSGroupChangePolicy: nil,
SeccompProfile: nil,
},
ImagePullSecrets: nil,
Hostname: "",
Subdomain: "",
Affinity: nil,
SchedulerName: "default-scheduler",
Tolerations: [
{
Key: "node.kubernetes.io/not-ready",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: 300,
},
{
Key: "node.kubernetes.io/unreachable",
Operator: "Exists",
Value: "",
Effect: "NoExecute",
TolerationSeconds: 300,
},
],
HostAliases: nil,
PriorityClassName: "",
Priority: 0,
DNSConfig: nil,
ReadinessGates: nil,
RuntimeClassName: nil,
EnableServiceLinks: true,
PreemptionPolicy: "PreemptLowerPriority",
Overhead: nil,
TopologySpreadConstraints: nil,
SetHostnameAsFQDN: nil,
OS: nil,
},
Status: {
Phase: "Pending",
Conditions: [
{
Type: "PodScheduled",
Status: "False",
LastProbeTime: {
Time: 0001-01-01T00:00:00Z,
},
LastTransitionTime: {
Time: 2022-05-25T07:01:58Z,
},
Reason: "SchedulerError",
Message: "running PreBind plugin \"VolumeBinding\": binding volumes: timed out waiting for the condition",
},
],
Message: "",
Reason: "",
NominatedNodeName: "",
HostIP: "",
PodIP: "",
PodIPs: nil,
StartTime: nil,
InitContainerStatuses: nil,
ContainerStatuses: nil,
QOSClass: "BestEffort",
EphemeralContainerStatuses: nil,
},
}
May 25 07:06:58.935: FAIL: Unexpected error:
<*pod.timeoutError | 0xc000609840>: {
msg: "timed out while waiting for pod snapshotting-1544/restored-pvc-tester-gb99j to be running",
}
timed out while waiting for pod snapshotting-1544/restored-pvc-tester-gb99j to be running
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2()
test/e2e/storage/testsuites/snapshottable.go:410 +0x1685
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
[AfterEach] volume snapshot controller
test/e2e/storage/testsuites/snapshottable.go:172
May 25 07:06:59.035: INFO: Pod restored-pvc-tester-gb99j has the following logs:
May 25 07:06:59.035: INFO: Deleting pod "restored-pvc-tester-gb99j" in namespace "snapshotting-1544"
May 25 07:06:59.140: INFO: Wait up to 5m0s for pod "restored-pvc-tester-gb99j" to be fully deleted
May 25 07:06:59.231: INFO: deleting claim "snapshotting-1544"/"pvc-xmpc5"
May 25 07:06:59.334: INFO: deleting snapshot "snapshotting-1544"/"snapshot-dg5l9"
May 25 07:06:59.394: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-9c62e145-207c-4815-a076-4c266b25baec readyToUse:false]
May 25 07:06:59.394: INFO: snapshotContentName snapcontent-9c62e145-207c-4815-a076-4c266b25baec
May 25 07:06:59.524: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec to be deleted
May 25 07:06:59.565: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:01.608: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:03.650: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:05.693: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:07.736: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:09.779: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:11.821: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:13.867: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:15.910: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:17.953: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:19.996: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:22.039: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:24.082: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:26.125: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:28.168: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:30.212: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:32.263: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:34.306: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:36.349: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:38.391: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:40.435: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:42.478: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:44.521: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:46.564: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:48.627: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:50.670: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:52.712: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:54.755: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:56.798: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:07:58.840: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:00.887: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:02.930: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:04.977: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:07.021: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:09.064: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:11.106: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:13.156: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:15.200: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:17.241: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:19.286: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:21.338: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:23.381: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:25.424: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:27.467: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:29.510: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:31.553: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:33.595: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:35.637: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:37.681: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:39.723: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:41.765: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:43.808: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:45.850: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:47.893: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:49.937: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:51.980: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:54.022: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:56.066: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:08:58.109: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:00.166: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:02.209: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:04.256: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:06.299: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:08.343: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:10.386: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:12.429: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:14.472: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:16.513: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:18.556: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:20.607: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:22.650: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:24.693: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:26.738: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:28.812: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:30.856: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:32.900: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:34.942: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:36.985: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:39.027: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:41.070: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:43.113: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:45.161: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:47.205: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:49.249: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:51.291: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:53.336: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:55.379: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:57.422: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:09:59.465: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:01.509: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:03.551: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:05.607: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:07.649: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:09.692: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:11.736: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:13.778: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:15.821: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:17.870: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:19.914: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:21.957: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:23.999: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:26.042: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:28.088: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:30.136: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:32.178: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:34.221: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:36.264: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:38.307: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:40.349: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:42.392: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:44.437: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:46.479: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:48.522: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:50.567: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:52.613: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:54.656: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:56.699: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:10:58.742: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:00.785: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:02.828: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:04.874: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:06.917: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:08.961: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:11.016: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:13.058: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:15.103: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:17.146: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:19.188: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:21.233: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:23.276: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:25.319: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:27.362: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:29.404: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:31.448: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:33.495: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:35.536: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:37.580: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:39.623: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:41.666: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:43.709: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:45.751: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:47.794: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:49.837: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:51.880: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:53.925: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:55.966: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:11:58.010: INFO: volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec has been found and is not deleted
May 25 07:12:00.011: INFO: WaitUntil failed after reaching the timeout 5m0s
May 25 07:12:00.012: FAIL: Unexpected error:
<*errors.errorString | 0xc0018bdcc0>: {
s: "volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec is not deleted within 5m0s",
}
volumesnapshotcontents snapcontent-9c62e145-207c-4815-a076-4c266b25baec is not deleted within 5m0s
occurred
Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/framework.(*SnapshotResource).CleanupResource(0xc002444160, 0xc000fc1e00)
test/e2e/storage/framework/snapshot_resource.go:204 +0xefd
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.4.2.2()
test/e2e/storage/testsuites/snapshottable.go:361 +0x28
k8s.io/kubernetes/test/e2e/storage/utils.TryFunc(0xc000400630?)
test/e2e/storage/utils/utils.go:714 +0x6d
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.2()
test/e2e/storage/testsuites/snapshottable.go:167 +0x38
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.3()
test/e2e/storage/testsuites/snapshottable.go:173 +0x1b
k8s.io/kubernetes/test/e2e.RunE2ETests(0x256e617?)
test/e2e/e2e.go:130 +0x686
k8s.io/kubernetes/test/e2e.TestE2E(0xc000d86000?)
test/e2e/e2e_test.go:136 +0x19
testing.tRunner(0xc000614340, 0x73d5a60)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
�[1mSTEP�[0m: Deleting pvc
May 25 07:12:00.182: INFO: Deleting PersistentVolumeClaim "pd.csi.storage.gke.ionhbxr"
May 25 07:12:00.285: INFO: Waiting up to 5m0s for PersistentVolume pvc-32744921-51d9-4f41-97dd-edd18c268f43 to get deleted
May 25 07:12:00.404: INFO: PersistentVolume pvc-32744921-51d9-4f41-97dd-edd18c268f43 found and phase=Released (118.252159ms)
May 25 07:12:05.484: INFO: PersistentVolume pvc-32744921-51d9-4f41-97dd-edd18c268f43 was removed
�[1mSTEP�[0m: Deleting sc
�[1mSTEP�[0m: deleting the test namespace: snapshotting-1544
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-1544".
�[1mSTEP�[0m: Found 21 events.
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:22 +0000 UTC - event for pd.csi.storage.gke.ionhbxr: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:23 +0000 UTC - event for pd.csi.storage.gke.ionhbxr: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_f079b6d5-fe0e-4705-abaf-2caf6de15ed6 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-1544/pd.csi.storage.gke.ionhbxr"
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:23 +0000 UTC - event for pd.csi.storage.gke.ionhbxr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:26 +0000 UTC - event for pd.csi.storage.gke.ionhbxr: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_f079b6d5-fe0e-4705-abaf-2caf6de15ed6 } ProvisioningSucceeded: Successfully provisioned volume pvc-32744921-51d9-4f41-97dd-edd18c268f43
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:27 +0000 UTC - event for pvc-snapshottable-tester-xgjdr: {default-scheduler } Scheduled: Successfully assigned snapshotting-1544/pvc-snapshottable-tester-xgjdr to bootstrap-e2e-minion-group-gw73
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:36 +0000 UTC - event for pvc-snapshottable-tester-xgjdr: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-32744921-51d9-4f41-97dd-edd18c268f43"
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:38 +0000 UTC - event for pvc-snapshottable-tester-xgjdr: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" already present on machine
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:38 +0000 UTC - event for pvc-snapshottable-tester-xgjdr: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container volume-tester
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:38 +0000 UTC - event for pvc-snapshottable-tester-xgjdr: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container volume-tester
May 25 07:12:05.683: INFO: At 2022-05-25 06:46:43 +0000 UTC - event for snapshot-dg5l9: {snapshot-controller } CreatingSnapshot: Waiting for a snapshot snapshotting-1544/snapshot-dg5l9 to be created by the CSI driver.
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:44 +0000 UTC - event for pvc-snapshottable-data-tester-qb2xk: {default-scheduler } Scheduled: Successfully assigned snapshotting-1544/pvc-snapshottable-data-tester-qb2xk to bootstrap-e2e-minion-group-gw73
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:53 +0000 UTC - event for pvc-snapshottable-data-tester-qb2xk: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-32744921-51d9-4f41-97dd-edd18c268f43"
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:54 +0000 UTC - event for pvc-snapshottable-data-tester-qb2xk: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" already present on machine
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:55 +0000 UTC - event for pvc-snapshottable-data-tester-qb2xk: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container volume-tester
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:55 +0000 UTC - event for pvc-snapshottable-data-tester-qb2xk: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container volume-tester
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:58 +0000 UTC - event for pvc-xmpc5: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:58 +0000 UTC - event for pvc-xmpc5: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_f079b6d5-fe0e-4705-abaf-2caf6de15ed6 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-1544/pvc-xmpc5"
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:58 +0000 UTC - event for pvc-xmpc5: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_f079b6d5-fe0e-4705-abaf-2caf6de15ed6 } ProvisioningFailed: failed to provision volume with StorageClass "snapshotting-1544556c6": error getting handle for DataSource Type VolumeSnapshot by Name snapshot-dg5l9: error getting snapshot snapshot-dg5l9 from api server: the server could not find the requested resource (get volumesnapshots.snapshot.storage.k8s.io snapshot-dg5l9)
May 25 07:12:05.683: INFO: At 2022-05-25 06:51:58 +0000 UTC - event for pvc-xmpc5: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
May 25 07:12:05.683: INFO: At 2022-05-25 07:01:58 +0000 UTC - event for restored-pvc-tester-gb99j: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
May 25 07:12:05.683: INFO: At 2022-05-25 07:06:59 +0000 UTC - event for restored-pvc-tester-gb99j: {default-scheduler } FailedScheduling: running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "restored-pvc-tester-gb99j" not found
May 25 07:12:05.781: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 07:12:05.781: INFO:
May 25 07:12:05.883: INFO:
Logging node info for node bootstrap-e2e-master
May 25 07:12:05.983: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 17452 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-master"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 07:07:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:05 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:05 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:05 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:07:05 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:05.983: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 07:12:06.081: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 07:12:06.246: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container kube-apiserver ready: true, restart count 0
May 25 07:12:06.246: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container kube-scheduler ready: true, restart count 0
May 25 07:12:06.246: INFO: csi-gce-pd-node-9rrp5 started at 2022-05-25 06:46:18 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:06.246: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:12:06.246: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:12:06.246: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container etcd-container ready: true, restart count 0
May 25 07:12:06.246: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 07:12:06.246: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container etcd-container ready: true, restart count 0
May 25 07:12:06.246: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 07:12:06.246: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 07:12:06.246: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:06.246: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:06.246: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:06.246: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.246: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 07:12:06.607: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 07:12:06.607: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 07:12:06.683: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 17666 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-gw73 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-gw73"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 06:51:53 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 07:07:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 07:08:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:06.683: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 07:12:06.781: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 07:12:06.906: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.906: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:12:06.906: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:06.906: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:06.906: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:06.906: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.906: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:12:06.906: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:06.906: INFO: Container coredns ready: true, restart count 0
May 25 07:12:06.906: INFO: csi-gce-pd-node-bll82 started at 2022-05-25 06:46:18 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:06.906: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:12:06.906: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:12:07.227: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 07:12:07.228: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 07:12:07.282: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 17979 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-kn6c topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-kn6c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:18:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:08:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:10:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:07.284: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 07:12:07.382: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 07:12:07.504: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:07.504: INFO: Container autoscaler ready: true, restart count 0
May 25 07:12:07.504: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:07.504: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 07:12:07.504: INFO: csi-gce-pd-node-wvz7g started at 2022-05-25 06:46:18 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:07.504: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:12:07.504: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:12:07.504: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:07.504: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:07.504: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:07.504: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:07.504: INFO: Container coredns ready: true, restart count 0
May 25 07:12:07.504: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:07.504: INFO: Container default-http-backend ready: true, restart count 0
May 25 07:12:07.504: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:07.504: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:12:07.504: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:07.504: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:12:07.869: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 07:12:07.869: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 07:12:07.938: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 18190 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-nndt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-nndt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:45:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:08:23 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:11:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:07.938: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 07:12:08.035: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 07:12:08.163: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:08.163: INFO: Container metrics-server ready: true, restart count 0
May 25 07:12:08.163: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 07:12:08.163: INFO: csi-gce-pd-controller-0 started at 2022-05-25 06:46:18 +0000 UTC (0+5 container statuses recorded)
May 25 07:12:08.163: INFO: Container csi-attacher ready: true, restart count 0
May 25 07:12:08.163: INFO: Container csi-provisioner ready: true, restart count 0
May 25 07:12:08.163: INFO: Container csi-resizer ready: true, restart count 0
May 25 07:12:08.163: INFO: Container csi-snapshotter ready: true, restart count 0
May 25 07:12:08.163: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:12:08.163: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:08.163: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:12:08.163: INFO: csi-gce-pd-node-824jg started at 2022-05-25 06:46:18 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:08.163: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:12:08.163: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:12:08.163: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:08.163: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:08.163: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:08.163: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:08.163: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:12:08.511: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-1544] to vanish
�[1mSTEP�[0m: uninstalling csi gce-pd driver
May 25 07:12:14.633: INFO: deleting *v1.ServiceAccount: snapshotting-1544-2274/csi-attacher
May 25 07:12:14.736: INFO: deleting *v1.ClusterRole: external-attacher-runner-snapshotting-1544
May 25 07:12:14.835: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-1544
May 25 07:12:14.933: INFO: deleting *v1.Role: snapshotting-1544-2274/external-attacher-cfg-snapshotting-1544
May 25 07:12:15.034: INFO: deleting *v1.RoleBinding: snapshotting-1544-2274/csi-attacher-role-cfg
May 25 07:12:15.140: INFO: deleting *v1.ServiceAccount: snapshotting-1544-2274/csi-provisioner
May 25 07:12:15.235: INFO: deleting *v1.ClusterRole: external-provisioner-runner-snapshotting-1544
May 25 07:12:15.333: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-1544
May 25 07:12:15.433: INFO: deleting *v1.Role: snapshotting-1544-2274/external-provisioner-cfg-snapshotting-1544
May 25 07:12:15.533: INFO: deleting *v1.RoleBinding: snapshotting-1544-2274/csi-provisioner-role-cfg
May 25 07:12:15.634: INFO: deleting *v1.ServiceAccount: snapshotting-1544-2274/csi-gce-pd-controller-sa
May 25 07:12:15.732: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-1544
May 25 07:12:15.834: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-1544
May 25 07:12:15.934: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-1544
May 25 07:12:16.034: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-1544
May 25 07:12:16.134: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-1544
May 25 07:12:16.300: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-1544
May 25 07:12:16.349: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-1544
May 25 07:12:16.434: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-1544
May 25 07:12:16.536: INFO: deleting *v1.Role: snapshotting-1544-2274/csi-gce-pd-leaderelection-role-snapshotting-1544
May 25 07:12:16.634: INFO: deleting *v1.RoleBinding: snapshotting-1544-2274/csi-gce-pd-controller-leaderelection-binding
May 25 07:12:16.734: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-1544
May 25 07:12:16.833: INFO: deleting *v1.DaemonSet: snapshotting-1544-2274/csi-gce-pd-node
May 25 07:12:16.934: INFO: deleting *v1.StatefulSet: snapshotting-1544-2274/csi-gce-pd-controller
�[1mSTEP�[0m: deleting the driver namespace: snapshotting-1544-2274
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-1544-2274".
�[1mSTEP�[0m: Found 71 events.
May 25 07:12:17.134: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-controller: {statefulset-controller } SuccessfulCreate: create Pod csi-gce-pd-controller-0 in StatefulSet csi-gce-pd-controller successful
May 25 07:12:17.134: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-controller-0: {default-scheduler } Scheduled: Successfully assigned snapshotting-1544-2274/csi-gce-pd-controller-0 to bootstrap-e2e-minion-group-nndt
May 25 07:12:17.134: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-9rrp5
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-824jg
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-bll82
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-wvz7g
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node-824jg: {default-scheduler } Scheduled: Successfully assigned snapshotting-1544-2274/csi-gce-pd-node-824jg to bootstrap-e2e-minion-group-nndt
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node-9rrp5: {default-scheduler } Scheduled: Successfully assigned snapshotting-1544-2274/csi-gce-pd-node-9rrp5 to bootstrap-e2e-master
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node-bll82: {default-scheduler } Scheduled: Successfully assigned snapshotting-1544-2274/csi-gce-pd-node-bll82 to bootstrap-e2e-minion-group-gw73
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:18 +0000 UTC - event for csi-gce-pd-node-wvz7g: {default-scheduler } Scheduled: Successfully assigned snapshotting-1544-2274/csi-gce-pd-node-wvz7g to bootstrap-e2e-minion-group-kn6c
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-m6t4z" : failed to sync configmap cache: timed out waiting for the condition
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-pkr5z" : failed to sync configmap cache: timed out waiting for the condition
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Created: Created container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Created: Created container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Started: Started container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Started: Started container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:19 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-hct8t" : failed to sync configmap cache: timed out waiting for the condition
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3"
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:20 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-resizer
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-resizer
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-attacher
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-attacher
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-provisioner
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-snapshotter
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-snapshotter
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" in 235.597046ms
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0" already present on machine
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-provisioner
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:21 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:22 +0000 UTC - event for external-attacher-leader-pd-csi-storage-gke-io: {external-attacher-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:22 +0000 UTC - event for external-resizer-pd-csi-storage-gke-io: {external-resizer-pd-csi-storage-gke-io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:22 +0000 UTC - event for external-snapshotter-leader-pd-csi-storage-gke-io: {external-snapshotter-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:12:17.135: INFO: At 2022-05-25 06:46:22 +0000 UTC - event for pd-csi-storage-gke-io: {pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Killing: Stopping container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-9rrp5: {kubelet bootstrap-e2e-master} Killing: Stopping container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Killing: Stopping container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} Killing: Stopping container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Killing: Stopping container csi-driver-registrar
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:16 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} Killing: Stopping container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:17 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container gce-pd-driver
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:17 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-snapshotter
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:17 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-resizer
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:17 +0000 UTC - event for csi-gce-pd-node-824jg: {kubelet bootstrap-e2e-minion-group-nndt} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-824jg_snapshotting-1544-2274(121ef8d2-7626-48ce-97e8-e46351b99d75)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "5244f9e52c4778d2fabe667abc81982c58a92dca17891aa04a011be177bc100d": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:17 +0000 UTC - event for csi-gce-pd-node-bll82: {kubelet bootstrap-e2e-minion-group-gw73} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-bll82_snapshotting-1544-2274(52df6e2c-e857-431a-938e-a801f032ef61)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "d03ba2d571a605afaa699ae01af0f48ab493625f017b82cb994ec07bcded6795": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 07:12:17.135: INFO: At 2022-05-25 07:12:17 +0000 UTC - event for csi-gce-pd-node-wvz7g: {kubelet bootstrap-e2e-minion-group-kn6c} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-wvz7g_snapshotting-1544-2274(7bc80e0f-7328-4485-a085-11ba0a738609)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "3559e0f633b94d48d1a6b42b5dd3f2060cc76488171d28a833b6f03257214601": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 07:12:17.196: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 07:12:17.196: INFO: csi-gce-pd-controller-0 bootstrap-e2e-minion-group-nndt Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:22 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:22 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC }]
May 25 07:12:17.196: INFO: csi-gce-pd-node-824jg bootstrap-e2e-minion-group-nndt Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:21 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:21 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC }]
May 25 07:12:17.196: INFO: csi-gce-pd-node-9rrp5 bootstrap-e2e-master Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:20 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:20 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC }]
May 25 07:12:17.196: INFO: csi-gce-pd-node-bll82 bootstrap-e2e-minion-group-gw73 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:19 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:19 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC }]
May 25 07:12:17.196: INFO: csi-gce-pd-node-wvz7g bootstrap-e2e-minion-group-kn6c Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:21 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:21 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 06:46:18 +0000 UTC }]
May 25 07:12:17.196: INFO:
May 25 07:12:17.728: INFO: Unable to fetch snapshotting-1544-2274/csi-gce-pd-node-bll82/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-bll82)
May 25 07:12:17.770: INFO: Unable to fetch snapshotting-1544-2274/csi-gce-pd-node-bll82/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-bll82)
May 25 07:12:17.811: INFO: Unable to fetch snapshotting-1544-2274/csi-gce-pd-node-wvz7g/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-wvz7g)
May 25 07:12:17.852: INFO: Unable to fetch snapshotting-1544-2274/csi-gce-pd-node-wvz7g/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-wvz7g)
May 25 07:12:17.897: INFO:
Logging node info for node bootstrap-e2e-master
May 25 07:12:17.940: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 18380 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 07:12:11 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:12:11 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:12:11 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:12:11 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:12:11 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:17.940: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 07:12:17.983: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 07:12:18.044: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 07:12:18.044: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:18.044: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:18.044: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:18.044: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 07:12:18.044: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 07:12:18.044: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container etcd-container ready: true, restart count 0
May 25 07:12:18.044: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 07:12:18.044: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container etcd-container ready: true, restart count 0
May 25 07:12:18.044: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container kube-apiserver ready: true, restart count 0
May 25 07:12:18.044: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.044: INFO: Container kube-scheduler ready: true, restart count 0
May 25 07:12:18.044: INFO: csi-gce-pd-node-9rrp5 started at 2022-05-25 06:46:18 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:18.044: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:12:18.044: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:12:18.253: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 07:12:18.253: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 07:12:18.295: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 18378 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-gw73 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 06:51:53 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-05-25 07:07:17 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-05-25 07:08:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:08:21 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:07:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:18.295: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 07:12:18.336: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 07:12:18.385: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.385: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:12:18.385: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:18.385: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:18.385: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:18.385: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.385: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:12:18.385: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.385: INFO: Container coredns ready: true, restart count 0
May 25 07:12:18.568: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 07:12:18.568: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 07:12:18.610: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 18370 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-kn6c topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:18:51 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:08:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:10:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:08:17 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:10:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:18.611: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 07:12:18.651: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 07:12:18.701: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.701: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:12:18.701: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.701: INFO: Container default-http-backend ready: true, restart count 0
May 25 07:12:18.701: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.701: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:12:18.701: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:18.701: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:18.701: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:18.701: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.701: INFO: Container coredns ready: true, restart count 0
May 25 07:12:18.701: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.701: INFO: Container autoscaler ready: true, restart count 0
May 25 07:12:18.701: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:18.701: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 07:12:18.881: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 07:12:18.881: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 07:12:18.924: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 18382 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-nndt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:45:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:08:23 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:11:30 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:08:23 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:11:30 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:12:18.924: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 07:12:18.965: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 07:12:19.014: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:19.014: INFO: Container metrics-server ready: true, restart count 0
May 25 07:12:19.014: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 07:12:19.014: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:19.014: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:12:19.014: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:12:19.014: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:12:19.014: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:12:19.014: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 07:12:19.014: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:12:19.187: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-1544-2274] to vanish
[AfterEach] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource]
test/e2e/framework/framework.go:187
May 25 07:12:25.287: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sCSI\sVolumes\s\[Driver\:\spd\.csi\.storage\.gke\.io\]\[Serial\]\s\[Testpattern\:\sPre\-provisioned\sSnapshot\s\(retain\spolicy\)\]\ssnapshottable\[Feature\:VolumeSnapshotDataSource\]\svolume\ssnapshot\scontroller\s\sshould\scheck\ssnapshot\sfields\,\scheck\srestore\scorrectly\sworks\safter\smodifying\ssource\sdata\,\scheck\sdeletion\s\(persistent\)$'
test/e2e/storage/testsuites/snapshottable.go:278
May 25 07:34:56.141: Unexpected error:
<*errors.errorString | 0xc0030cb100>: {
s: "VolumeSnapshot snapshot-tvr66 is not ready within 5m0s",
}
VolumeSnapshot snapshot-tvr66 is not ready within 5m0s
occurred
test/e2e/storage/utils/snapshot.go:86
May 25 07:37:48.262: INFO: Running AfterSuite actions on all nodes
May 25 07:37:48.262: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage/drivers.generateDriverCleanupFunc.func1
�[1mSTEP�[0m: deleting the test namespace: snapshotting-5728
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-5728".
�[1mSTEP�[0m: Found 10 events.
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:36 +0000 UTC - event for pd.csi.storage.gke.io9xdzv: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:36 +0000 UTC - event for pd.csi.storage.gke.io9xdzv: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:36 +0000 UTC - event for pd.csi.storage.gke.io9xdzv: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_3e21bfd5-e8ef-4959-b85c-c35d74b985b5 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-5728/pd.csi.storage.gke.io9xdzv"
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:40 +0000 UTC - event for pd.csi.storage.gke.io9xdzv: {pd.csi.storage.gke.io_bootstrap-e2e-minion-group-nndt_3e21bfd5-e8ef-4959-b85c-c35d74b985b5 } ProvisioningSucceeded: Successfully provisioned volume pvc-5405d6f7-ae15-4cb3-b24b-e949caffccb0
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:40 +0000 UTC - event for pvc-snapshottable-tester-6svlr: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728/pvc-snapshottable-tester-6svlr to bootstrap-e2e-minion-group-gw73
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:50 +0000 UTC - event for pvc-snapshottable-tester-6svlr: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-5405d6f7-ae15-4cb3-b24b-e949caffccb0"
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:51 +0000 UTC - event for pvc-snapshottable-tester-6svlr: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" already present on machine
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:51 +0000 UTC - event for pvc-snapshottable-tester-6svlr: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container volume-tester
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:51 +0000 UTC - event for pvc-snapshottable-tester-6svlr: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container volume-tester
May 25 07:37:48.303: INFO: At 2022-05-25 07:29:55 +0000 UTC - event for snapshot-tvr66: {snapshot-controller } CreatingSnapshot: Waiting for a snapshot snapshotting-5728/snapshot-tvr66 to be created by the CSI driver.
May 25 07:37:48.345: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 07:37:48.345: INFO:
May 25 07:37:48.388: INFO:
Logging node info for node bootstrap-e2e-master
May 25 07:37:48.431: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 22370 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-master"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 07:37:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:37:48.431: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 07:37:48.472: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 07:37:48.561: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container etcd-container ready: true, restart count 0
May 25 07:37:48.561: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 07:37:48.561: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 07:37:48.561: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:48.561: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:37:48.561: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:37:48.561: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 07:37:48.561: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 07:37:48.561: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container kube-scheduler ready: true, restart count 0
May 25 07:37:48.561: INFO: csi-gce-pd-node-khpnk started at 2022-05-25 07:29:32 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:48.561: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:37:48.561: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:37:48.561: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container etcd-container ready: true, restart count 0
May 25 07:37:48.561: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.561: INFO: Container kube-apiserver ready: true, restart count 0
May 25 07:37:48.825: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 07:37:48.825: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 07:37:48.870: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 21922 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-gw73 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-gw73"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 07:29:50 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:35:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:37:48.870: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 07:37:48.912: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 07:37:48.982: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.982: INFO: Container coredns ready: true, restart count 0
May 25 07:37:48.982: INFO: csi-gce-pd-node-w49h2 started at 2022-05-25 07:29:32 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:48.982: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:37:48.982: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:37:48.982: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.982: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:37:48.982: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:48.982: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:37:48.982: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:37:48.982: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:48.982: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:37:49.166: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 07:37:49.167: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 07:37:49.209: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 21790 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-kn6c topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-kn6c"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 07:13:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:37:49.211: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 07:37:49.252: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 07:37:49.386: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.386: INFO: Container default-http-backend ready: true, restart count 0
May 25 07:37:49.386: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.386: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:37:49.386: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.386: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:37:49.386: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.386: INFO: Container coredns ready: true, restart count 0
May 25 07:37:49.386: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.386: INFO: Container autoscaler ready: true, restart count 0
May 25 07:37:49.386: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.386: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 07:37:49.386: INFO: csi-gce-pd-node-xf8h8 started at 2022-05-25 07:29:32 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:49.386: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:37:49.386: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:37:49.386: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:49.386: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:37:49.386: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:37:49.597: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 07:37:49.598: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 07:37:49.641: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 21887 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-nndt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-minion-group-nndt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:45:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 k8s.gcr.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:37:49.643: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 07:37:49.684: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 07:37:49.751: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.752: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:37:49.752: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:49.752: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:37:49.752: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:37:49.752: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 07:37:49.752: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:37:49.752: INFO: csi-gce-pd-controller-0 started at 2022-05-25 07:29:32 +0000 UTC (0+5 container statuses recorded)
May 25 07:37:49.752: INFO: Container csi-attacher ready: true, restart count 0
May 25 07:37:49.752: INFO: Container csi-provisioner ready: true, restart count 0
May 25 07:37:49.752: INFO: Container csi-resizer ready: true, restart count 0
May 25 07:37:49.752: INFO: Container csi-snapshotter ready: true, restart count 0
May 25 07:37:49.752: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:37:49.752: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:49.752: INFO: Container metrics-server ready: true, restart count 0
May 25 07:37:49.752: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 07:37:49.752: INFO: csi-gce-pd-node-5wpgm started at 2022-05-25 07:29:32 +0000 UTC (0+2 container statuses recorded)
May 25 07:37:49.752: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:37:49.752: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:37:49.975: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-5728] to vanish
May 25 07:37:52.872: INFO: PersistentVolume pvc-5405d6f7-ae15-4cb3-b24b-e949caffccb0 found and phase=Bound (2m56.595270832s)
May 25 07:37:57.915: INFO: PersistentVolume pvc-5405d6f7-ae15-4cb3-b24b-e949caffccb0 found and phase=Released (3m1.638010346s)
�[1mSTEP�[0m: uninstalling csi gce-pd driver
May 25 07:38:02.068: INFO: deleting *v1.ServiceAccount: snapshotting-5728-3780/csi-attacher
May 25 07:38:02.112: INFO: deleting *v1.ClusterRole: external-attacher-runner-snapshotting-5728
May 25 07:38:02.156: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-5728
May 25 07:38:02.200: INFO: deleting *v1.Role: snapshotting-5728-3780/external-attacher-cfg-snapshotting-5728
May 25 07:38:02.248: INFO: deleting *v1.RoleBinding: snapshotting-5728-3780/csi-attacher-role-cfg
May 25 07:38:02.293: INFO: deleting *v1.ServiceAccount: snapshotting-5728-3780/csi-provisioner
May 25 07:38:02.337: INFO: deleting *v1.ClusterRole: external-provisioner-runner-snapshotting-5728
May 25 07:38:02.393: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-5728
May 25 07:38:02.435: INFO: deleting *v1.Role: snapshotting-5728-3780/external-provisioner-cfg-snapshotting-5728
May 25 07:38:02.480: INFO: deleting *v1.RoleBinding: snapshotting-5728-3780/csi-provisioner-role-cfg
May 25 07:38:02.525: INFO: deleting *v1.ServiceAccount: snapshotting-5728-3780/csi-gce-pd-controller-sa
May 25 07:38:02.569: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-5728
May 25 07:38:02.612: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-5728
May 25 07:38:02.656: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-5728
May 25 07:38:02.701: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-5728
May 25 07:38:02.745: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-5728
May 25 07:38:02.789: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-5728
May 25 07:38:02.833: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-5728
May 25 07:38:02.880: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-5728
May 25 07:38:02.924: INFO: deleting *v1.Role: snapshotting-5728-3780/csi-gce-pd-leaderelection-role-snapshotting-5728
May 25 07:38:02.957: INFO: PersistentVolume pvc-5405d6f7-ae15-4cb3-b24b-e949caffccb0 was removed
�[1mSTEP�[0m: Deleting sc
May 25 07:38:02.970: INFO: deleting *v1.RoleBinding: snapshotting-5728-3780/csi-gce-pd-controller-leaderelection-binding
�[1mSTEP�[0m: deleting the test namespace: snapshotting-5728
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-5728".
May 25 07:38:03.015: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-5728
�[1mSTEP�[0m: Found 0 events.
May 25 07:38:03.060: INFO: deleting *v1.DaemonSet: snapshotting-5728-3780/csi-gce-pd-node
May 25 07:38:03.084: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 07:38:03.085: INFO:
May 25 07:38:03.107: INFO: deleting *v1.StatefulSet: snapshotting-5728-3780/csi-gce-pd-controller
May 25 07:38:03.153: INFO:
Logging node info for node bootstrap-e2e-master
�[1mSTEP�[0m: deleting the driver namespace: snapshotting-5728-3780
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-5728-3780".
May 25 07:38:03.240: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 22370 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"pd.csi.storage.gke.io":"projects/k8s-jkns-e2e-protobuf/zones/us-west1-b/instances/bootstrap-e2e-master"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 07:37:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:03.241: INFO:
Logging kubelet events for node bootstrap-e2e-master
�[1mSTEP�[0m: Found 57 events.
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-controller: {statefulset-controller } SuccessfulCreate: create Pod csi-gce-pd-controller-0 in StatefulSet csi-gce-pd-controller successful
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-controller-0: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-controller-0 to bootstrap-e2e-minion-group-nndt
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-5wpgm
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-khpnk
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-xf8h8
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-w49h2
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-5wpgm: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-5wpgm to bootstrap-e2e-minion-group-nndt
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-khpnk: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-khpnk to bootstrap-e2e-master
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-w49h2: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-w49h2 to bootstrap-e2e-minion-group-gw73
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-xf8h8: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-xf8h8 to bootstrap-e2e-minion-group-kn6c
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-snapshotter
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-provisioner
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-snapshotter
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3"
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" in 352.898237ms
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-provisioner
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Started: Started container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Created: Created container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Started: Started container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Created: Created container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-resizer
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-resizer
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-attacher
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-attacher
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0" already present on machine
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for external-snapshotter-leader-pd-csi-storage-gke-io: {external-snapshotter-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:35 +0000 UTC - event for external-attacher-leader-pd-csi-storage-gke-io: {external-attacher-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:35 +0000 UTC - event for external-resizer-pd-csi-storage-gke-io: {external-resizer-pd-csi-storage-gke-io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:03.272: INFO: At 2022-05-25 07:29:35 +0000 UTC - event for pd-csi-storage-gke-io: {pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:03.272: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Killing: Stopping container csi-driver-registrar
May 25 07:38:03.272: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Killing: Stopping container csi-driver-registrar
May 25 07:38:03.307: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 07:38:03.325: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 07:38:03.325: INFO: csi-gce-pd-controller-0 bootstrap-e2e-minion-group-nndt Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:34 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:34 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC }]
May 25 07:38:03.325: INFO: csi-gce-pd-node-5wpgm bootstrap-e2e-minion-group-nndt Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:33 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:33 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC }]
May 25 07:38:03.325: INFO: csi-gce-pd-node-khpnk bootstrap-e2e-master Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:34 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:34 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC }]
May 25 07:38:03.325: INFO: csi-gce-pd-node-w49h2 bootstrap-e2e-minion-group-gw73 Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:33 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:33 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC }]
May 25 07:38:03.325: INFO: csi-gce-pd-node-xf8h8 bootstrap-e2e-minion-group-kn6c Running 30s [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:33 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:33 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-05-25 07:29:32 +0000 UTC }]
May 25 07:38:03.325: INFO:
May 25 07:38:03.384: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container etcd-container ready: true, restart count 0
May 25 07:38:03.384: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container kube-apiserver ready: true, restart count 0
May 25 07:38:03.384: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container kube-scheduler ready: true, restart count 0
May 25 07:38:03.384: INFO: csi-gce-pd-node-khpnk started at 2022-05-25 07:29:32 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:03.384: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:38:03.384: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:38:03.384: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 07:38:03.384: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:03.384: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:03.384: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:03.384: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 07:38:03.384: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 07:38:03.384: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container etcd-container ready: true, restart count 0
May 25 07:38:03.384: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:03.384: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 07:38:03.871: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 07:38:03.871: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 07:38:03.916: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 22484 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-gw73 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 07:29:50 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:35:00 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:03.916: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 07:38:03.959: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 07:38:04.062: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:04.062: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:04.062: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:04.062: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.062: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:04.062: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.062: INFO: Container coredns ready: true, restart count 0
May 25 07:38:04.062: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.062: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:04.165: INFO: Unable to fetch snapshotting-5728-3780/csi-gce-pd-node-w49h2/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-w49h2)
May 25 07:38:04.217: INFO: Unable to fetch snapshotting-5728-3780/csi-gce-pd-node-w49h2/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-w49h2)
May 25 07:38:04.259: INFO: Unable to fetch snapshotting-5728-3780/csi-gce-pd-node-xf8h8/csi-driver-registrar logs: the server could not find the requested resource (get pods csi-gce-pd-node-xf8h8)
May 25 07:38:04.270: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 07:38:04.270: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:04.301: INFO: Unable to fetch snapshotting-5728-3780/csi-gce-pd-node-xf8h8/gce-pd-driver logs: the server could not find the requested resource (get pods csi-gce-pd-node-xf8h8)
May 25 07:38:04.313: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 22495 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-kn6c topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 07:13:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:04.313: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:04.345: INFO:
Logging node info for node bootstrap-e2e-master
May 25 07:38:04.356: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 07:38:04.387: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 22489 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 07:37:41 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:04.388: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 07:38:04.414: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.414: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 07:38:04.414: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:04.414: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:04.414: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:04.414: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.414: INFO: Container coredns ready: true, restart count 0
May 25 07:38:04.414: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.414: INFO: Container autoscaler ready: true, restart count 0
May 25 07:38:04.414: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.414: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:04.414: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.414: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:04.414: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.414: INFO: Container default-http-backend ready: true, restart count 0
May 25 07:38:04.429: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 07:38:04.481: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container etcd-container ready: true, restart count 0
May 25 07:38:04.481: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container kube-apiserver ready: true, restart count 0
May 25 07:38:04.481: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container kube-scheduler ready: true, restart count 0
May 25 07:38:04.481: INFO: csi-gce-pd-node-khpnk started at 2022-05-25 07:29:32 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:04.481: INFO: Container csi-driver-registrar ready: true, restart count 0
May 25 07:38:04.481: INFO: Container gce-pd-driver ready: true, restart count 0
May 25 07:38:04.481: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 07:38:04.481: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 07:38:04.481: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container etcd-container ready: true, restart count 0
May 25 07:38:04.481: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 07:38:04.481: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.481: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 07:38:04.481: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:04.481: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:04.481: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:04.650: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:04.650: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 07:38:04.697: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 22494 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-nndt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:45:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 k8s.gcr.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:04.697: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 07:38:04.741: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 07:38:04.759: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 07:38:04.759: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 07:38:04.802: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 22484 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-gw73 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 07:29:50 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:35:00 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:04.803: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 07:38:04.842: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.842: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:04.842: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:04.842: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:04.842: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:04.842: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.842: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:04.842: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:04.842: INFO: Container metrics-server ready: true, restart count 0
May 25 07:38:04.842: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 07:38:04.853: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 07:38:04.994: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.994: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:04.994: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:04.994: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:04.994: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:04.994: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.994: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:04.994: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:04.994: INFO: Container coredns ready: true, restart count 0
May 25 07:38:05.102: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-5728] to vanish
May 25 07:38:05.179: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 07:38:05.179: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:05.222: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 22495 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-kn6c topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 07:13:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:05.222: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:05.266: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 07:38:05.319: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.319: INFO: Container default-http-backend ready: true, restart count 0
May 25 07:38:05.319: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.319: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:05.319: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.319: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:05.319: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.319: INFO: Container autoscaler ready: true, restart count 0
May 25 07:38:05.319: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.319: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 07:38:05.319: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:05.319: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:05.319: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:05.319: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.319: INFO: Container coredns ready: true, restart count 0
May 25 07:38:05.510: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:05.510: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 07:38:05.585: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 22494 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-nndt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:45:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 k8s.gcr.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:05.586: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 07:38:05.629: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 07:38:05.682: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:05.682: INFO: Container metrics-server ready: true, restart count 0
May 25 07:38:05.682: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 07:38:05.682: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.682: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:05.682: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:05.682: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:05.682: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:05.682: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:05.682: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:05.868: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-5728-3780] to vanish
�[1mSTEP�[0m: uninstalling csi gce-pd driver
May 25 07:38:07.188: INFO: deleting *v1.ServiceAccount: snapshotting-5728-3780/csi-attacher
May 25 07:38:07.230: INFO: deleting *v1.ClusterRole: external-attacher-runner-snapshotting-5728
May 25 07:38:07.272: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-snapshotting-5728
May 25 07:38:07.316: INFO: deleting *v1.Role: snapshotting-5728-3780/external-attacher-cfg-snapshotting-5728
May 25 07:38:07.361: INFO: deleting *v1.RoleBinding: snapshotting-5728-3780/csi-attacher-role-cfg
May 25 07:38:07.409: INFO: deleting *v1.ServiceAccount: snapshotting-5728-3780/csi-provisioner
May 25 07:38:07.453: INFO: deleting *v1.ClusterRole: external-provisioner-runner-snapshotting-5728
May 25 07:38:07.495: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-snapshotting-5728
May 25 07:38:07.537: INFO: deleting *v1.Role: snapshotting-5728-3780/external-provisioner-cfg-snapshotting-5728
May 25 07:38:07.578: INFO: deleting *v1.RoleBinding: snapshotting-5728-3780/csi-provisioner-role-cfg
May 25 07:38:07.620: INFO: deleting *v1.ServiceAccount: snapshotting-5728-3780/csi-gce-pd-controller-sa
May 25 07:38:07.663: INFO: deleting *v1.ClusterRole: csi-gce-pd-provisioner-role-snapshotting-5728
May 25 07:38:07.709: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-provisioner-binding-snapshotting-5728
May 25 07:38:07.753: INFO: deleting *v1.ClusterRole: csi-gce-pd-attacher-role-snapshotting-5728
May 25 07:38:07.797: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-attacher-binding-snapshotting-5728
May 25 07:38:07.839: INFO: deleting *v1.ClusterRole: csi-gce-pd-resizer-role-snapshotting-5728
May 25 07:38:07.881: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-resizer-binding-snapshotting-5728
May 25 07:38:07.923: INFO: deleting *v1.ClusterRole: csi-gce-pd-snapshotter-role-snapshotting-5728
May 25 07:38:07.969: INFO: deleting *v1.ClusterRoleBinding: csi-gce-pd-controller-snapshotter-binding-snapshotting-5728
May 25 07:38:08.013: INFO: deleting *v1.Role: snapshotting-5728-3780/csi-gce-pd-leaderelection-role-snapshotting-5728
May 25 07:38:08.058: INFO: deleting *v1.RoleBinding: snapshotting-5728-3780/csi-gce-pd-controller-leaderelection-binding
May 25 07:38:08.100: INFO: deleting *v1.ClusterRoleBinding: psp-csi-controller-driver-registrar-role-snapshotting-5728
May 25 07:38:08.141: INFO: deleting *v1.DaemonSet: snapshotting-5728-3780/csi-gce-pd-node
May 25 07:38:08.184: INFO: deleting *v1.StatefulSet: snapshotting-5728-3780/csi-gce-pd-controller
�[1mSTEP�[0m: deleting the driver namespace: snapshotting-5728-3780
�[1mSTEP�[0m: Collecting events from namespace "snapshotting-5728-3780".
�[1mSTEP�[0m: Found 71 events.
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-controller: {statefulset-controller } SuccessfulCreate: create Pod csi-gce-pd-controller-0 in StatefulSet csi-gce-pd-controller successful
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-controller-0: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-controller-0 to bootstrap-e2e-minion-group-nndt
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-5wpgm
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-khpnk
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-xf8h8
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node: {daemonset-controller } SuccessfulCreate: Created pod: csi-gce-pd-node-w49h2
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-5wpgm: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-5wpgm to bootstrap-e2e-minion-group-nndt
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-khpnk: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-khpnk to bootstrap-e2e-master
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-w49h2: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-w49h2 to bootstrap-e2e-minion-group-gw73
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-xf8h8: {default-scheduler } Scheduled: Successfully assigned snapshotting-5728-3780/csi-gce-pd-node-xf8h8 to bootstrap-e2e-minion-group-kn6c
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:32 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-snapshotter
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Successfully pulled image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3" in 352.898237ms
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-provisioner
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0" already present on machine
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-snapshotter
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulling: Pulling image "k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3"
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-attacher:v3.1.0" already present on machine
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-provisioner
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-driver-registrar
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-driver-registrar
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Started: Started container gce-pd-driver
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Created: Created container gce-pd-driver
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:08.270: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Created: Created container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Started: Started container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0" already present on machine
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Started: Started container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Created: Created container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Started: Started container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:33 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Created: Created container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-resizer
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-resizer
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/sig-storage/csi-resizer:v1.1.0" already present on machine
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Started: Started container csi-attacher
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Created: Created container csi-attacher
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Pulled: Container image "k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2" already present on machine
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:34 +0000 UTC - event for external-snapshotter-leader-pd-csi-storage-gke-io: {external-snapshotter-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:35 +0000 UTC - event for external-attacher-leader-pd-csi-storage-gke-io: {external-attacher-leader-pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:35 +0000 UTC - event for external-resizer-pd-csi-storage-gke-io: {external-resizer-pd-csi-storage-gke-io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:08.271: INFO: At 2022-05-25 07:29:35 +0000 UTC - event for pd-csi-storage-gke-io: {pd.csi.storage.gke.io/bootstrap-e2e-minion-group-nndt } LeaderElection: bootstrap-e2e-minion-group-nndt became leader
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-snapshotter
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-attacher
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-resizer
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-controller-0: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-provisioner
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-5wpgm_snapshotting-5728-3780(929857a9-faf6-491d-87e2-3b24ce1341f9)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "714143ede59393ec8c55afe81c76d07213c7d7f62578f2d477eb9fa0393c5638": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-5wpgm: {kubelet bootstrap-e2e-minion-group-nndt} Killing: Stopping container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-khpnk_snapshotting-5728-3780(14d768e5-1ec1-42ac-9508-d58add48c87c)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "5a0aef95b16a150d9128850eb1a36f1d9df528ef50c4cbca0df3cffe28cd75f7": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Killing: Stopping container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-khpnk: {kubelet bootstrap-e2e-master} Killing: Stopping container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Killing: Stopping container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} Killing: Stopping container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-w49h2: {kubelet bootstrap-e2e-minion-group-gw73} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-w49h2_snapshotting-5728-3780(9388053a-8e78-4e72-bea3-abbced274cf2)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "1d20e94e47fc0448f8356bc789ec711269e174ec537bc1e3c6ea77f7704ff5f4": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Killing: Stopping container gce-pd-driver
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} Killing: Stopping container csi-driver-registrar
May 25 07:38:08.271: INFO: At 2022-05-25 07:38:03 +0000 UTC - event for csi-gce-pd-node-xf8h8: {kubelet bootstrap-e2e-minion-group-kn6c} FailedPreStopHook: Exec lifecycle hook ([/bin/sh -c rm -rf /registration/pd.csi.storage.gke.io /registration/pd.csi.storage.gke.io-reg.sock]) for Container "csi-driver-registrar" in Pod "csi-gce-pd-node-xf8h8_snapshotting-5728-3780(33e55eaf-568a-44b4-8e6c-cf3ba42a8ae8)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "6f4f4d41ff8c6d74643ede326084656b247ed6bfa0f3b914f836bcdd01a179bc": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown, message: ""
May 25 07:38:08.317: INFO: POD NODE PHASE GRACE CONDITIONS
May 25 07:38:08.317: INFO:
May 25 07:38:08.361: INFO:
Logging node info for node bootstrap-e2e-master
May 25 07:38:08.405: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 62586b26-f318-4462-a65f-73d909ffb6fb 22489 0 2022-05-25 05:42:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.gke.io/zone:us-west1-b topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:42:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-05-25 05:43:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-05-25 07:37:41 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16684785664 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3864322048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{15016307073 0} {<nil>} 15016307073 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3602178048 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:06 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:42:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:37:41 +0000 UTC,LastTransitionTime:2022-05-25 05:43:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.82.64.164,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:07ffb543bedc149534b2440709dac943,SystemUUID:07ffb543-bedc-1495-34b2-440709dac943,BootID:ffad29f2-16bf-4ad1-ad1d-52719e57609a,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-apiserver-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:127273427,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:117127491,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:13f53ed1d91e2e11aac476ee9a0269fdda6cc4874eba903efd40daf50c55eee5 k8s.gcr.io/etcd:3.5.3-0],SizeBytes:102143581,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:50940226,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:be60ef505fc80879eeb5d8bf3ad8bb1146b395afc2394584645e99431806c26c gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.12.0],SizeBytes:32705362,},ContainerImage{Names:[k8s.gcr.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 k8s.gcr.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-server@sha256:d59ff043b173896cedeb897225ecfd2cdfc48591a04df035b439c04431421fc2 k8s.gcr.io/kas-network-proxy/proxy-server:v0.0.30],SizeBytes:17920446,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:08.405: INFO:
Logging kubelet events for node bootstrap-e2e-master
May 25 07:38:08.448: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
May 25 07:38:08.499: INFO: etcd-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container etcd-container ready: true, restart count 0
May 25 07:38:08.499: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container kube-apiserver ready: true, restart count 0
May 25 07:38:08.499: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container kube-scheduler ready: true, restart count 0
May 25 07:38:08.499: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container kube-addon-manager ready: true, restart count 0
May 25 07:38:08.499: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-05-25 05:42:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container l7-lb-controller ready: true, restart count 3
May 25 07:38:08.499: INFO: metadata-proxy-v0.1-h5slf started at 2022-05-25 05:43:22 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:08.499: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:08.499: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:08.499: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container konnectivity-server-container ready: true, restart count 0
May 25 07:38:08.499: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container kube-controller-manager ready: true, restart count 2
May 25 07:38:08.499: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-05-25 05:41:46 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.499: INFO: Container etcd-container ready: true, restart count 0
May 25 07:38:08.727: INFO:
Latency metrics for node bootstrap-e2e-master
May 25 07:38:08.728: INFO:
Logging node info for node bootstrap-e2e-minion-group-gw73
May 25 07:38:08.770: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gw73 53305ac9-099d-48bb-8f07-839959272f9f 22484 0 2022-05-25 05:43:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gw73 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-gw73 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-05-25 05:43:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-05-25 07:29:50 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:35:00 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-gw73,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:24 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:35:00 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.22.90,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gw73.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:ba28e223a7f95fc9dacbd25dfe5a248d,SystemUUID:ba28e223-a7f9-5fc9-dacb-d25dfe5a248d,BootID:0c4e9705-93db-49af-92aa-57d76f07bb44,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:08.770: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gw73
May 25 07:38:08.812: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gw73
May 25 07:38:08.870: INFO: kube-proxy-bootstrap-e2e-minion-group-gw73 started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.870: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:08.870: INFO: metadata-proxy-v0.1-xp6vg started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:08.870: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:08.870: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:08.870: INFO: konnectivity-agent-w8d6z started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.870: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:08.870: INFO: coredns-77c6689697-5hp2r started at 2022-05-25 05:43:27 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:08.870: INFO: Container coredns ready: true, restart count 0
May 25 07:38:09.047: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gw73
May 25 07:38:09.047: INFO:
Logging node info for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:09.089: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kn6c 5a1647aa-8463-4b13-b5e1-30519ce3e59d 22495 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kn6c kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-kn6c topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 07:13:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:13 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-kn6c,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821434880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559290880 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:19 +0000 UTC,LastTransitionTime:2022-05-25 05:43:09 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:13 +0000 UTC,LastTransitionTime:2022-05-25 05:43:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.105.122.208,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kn6c.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:02b1bfc3ed21c2df48d38dd9999b2d03,SystemUUID:02b1bfc3-ed21-c2df-48d3-8dd9999b2d03,BootID:4fe86ca6-64f9-4d84-8ec3-b4a76541e3f2,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/snapshot-controller@sha256:00fcc441ea9f72899c25eed61d602272a2a58c5f0014332bdcb5ac24acef08e4 k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0],SizeBytes:18952261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def k8s.gcr.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[k8s.gcr.io/coredns/coredns@sha256:5b6ec0d6de9baaf3e92d0f66cd96a25b9edbce8716f5f15dcd1a616b3abd590e k8s.gcr.io/coredns/coredns:v1.8.6],SizeBytes:13585107,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 k8s.gcr.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:09.090: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:09.131: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kn6c
May 25 07:38:09.183: INFO: l7-default-backend-5df94ffdcb-wp89r started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.183: INFO: Container default-http-backend ready: true, restart count 0
May 25 07:38:09.183: INFO: konnectivity-agent-9wwvp started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.183: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:09.183: INFO: kube-proxy-bootstrap-e2e-minion-group-kn6c started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.183: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:09.183: INFO: coredns-77c6689697-vvr4c started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.183: INFO: Container coredns ready: true, restart count 0
May 25 07:38:09.183: INFO: kube-dns-autoscaler-569c995844-2q7nt started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.183: INFO: Container autoscaler ready: true, restart count 0
May 25 07:38:09.183: INFO: volume-snapshot-controller-0 started at 2022-05-25 05:43:20 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.183: INFO: Container volume-snapshot-controller ready: true, restart count 0
May 25 07:38:09.183: INFO: metadata-proxy-v0.1-7h277 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:09.183: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:09.183: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:09.371: INFO:
Latency metrics for node bootstrap-e2e-minion-group-kn6c
May 25 07:38:09.371: INFO:
Logging node info for node bootstrap-e2e-minion-group-nndt
May 25 07:38:09.417: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-nndt c754c332-563f-43c4-a06c-6ae7dc9e97cd 22494 0 2022-05-25 05:43:05 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-nndt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.gke.io/zone:us-west1-b topology.hostpath.csi/node:bootstrap-e2e-minion-group-nndt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-05-25 05:43:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-05-25 06:45:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-05-25 07:33:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-05-25 07:34:49 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.gke.io/zone":{},"f:topology.hostpath.csi/node":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-protobuf/us-west1-b/bootstrap-e2e-minion-group-nndt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101241290752 0} {<nil>} BinarySI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7821426688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91117161526 0} {<nil>} 91117161526 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7559282688 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-05-25 07:33:26 +0000 UTC,LastTransitionTime:2022-05-25 05:43:10 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-05-25 05:43:20 +0000 UTC,LastTransitionTime:2022-05-25 05:43:20 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-25 07:34:49 +0000 UTC,LastTransitionTime:2022-05-25 05:43:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.199.164.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-nndt.c.k8s-jkns-e2e-protobuf.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:48f906a71ba572f2dd9da77d646f9c9e,SystemUUID:48f906a7-1ba5-72f2-dd9d-a77d646f9c9e,BootID:d99ed13e-3223-4ab1-93a1-f8790832a28e,KernelVersion:5.4.129+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.4.6,KubeletVersion:v1.25.0-alpha.0.620+d5579bf7786bac,KubeProxyVersion:v1.25.0-alpha.0.620+d5579bf7786bac,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/kube-proxy-amd64:v1.25.0-alpha.0.620_d5579bf7786bac],SizeBytes:110639159,},ContainerImage{Names:[k8s.gcr.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 k8s.gcr.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver@sha256:e9c355dd1ce57de91a0f9164bd2fe688938723e7f9bd40bb13652d603f41d25a k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.2],SizeBytes:73354282,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[k8s.gcr.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 k8s.gcr.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:6477988532358148d2e98f7c747db4e9250bbc7ad2664bf666348abf9ee1f5aa k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0],SizeBytes:22728994,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:6e0546563b18872b0aa0cad7255a26bb9a87cb879b7fc3e2383c867ef4f706fb k8s.gcr.io/sig-storage/csi-resizer:v1.3.0],SizeBytes:21671340,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:818f35653f2e214db81d655063e81995de9073328a3430498624c140881026a3 k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1],SizeBytes:21564520,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:80dec81b679a733fda448be92a2331150d99095947d04003ecff3dbd7f2a476a k8s.gcr.io/sig-storage/csi-attacher:v3.3.0],SizeBytes:21444261,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-provisioner@sha256:20c828075d1e36f679d6a91e905b0927141eef5e15be0c9a1ca4a6a0ed9313d2 k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0],SizeBytes:21205045,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-attacher@sha256:50c3cfd458fc8e0bf3c8c521eac39172009382fc66dc5044a330d137c6ed0b09 k8s.gcr.io/sig-storage/csi-attacher:v3.1.0],SizeBytes:20103959,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-resizer@sha256:7a5ba58a44e0d749e0767e4e37315bcf6a61f33ce3185c1991848af4db0fb70a k8s.gcr.io/sig-storage/csi-resizer:v1.1.0],SizeBytes:20096832,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-snapshotter@sha256:9af9bf28430b00a0cedeb2ec29acadce45e6afcecd8bdf31c793c624cfa75fa7 k8s.gcr.io/sig-storage/csi-snapshotter:v3.0.3],SizeBytes:19500777,},ContainerImage{Names:[k8s.gcr.io/sig-storage/hostpathplugin@sha256:6029c252dae6178c99b580de72d7776158edbc81be0de15cedc4152a3acfed18 k8s.gcr.io/sig-storage/hostpathplugin:v1.7.3],SizeBytes:15224494,},ContainerImage{Names:[k8s.gcr.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 k8s.gcr.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:a61d309da54641db41fb8f35718f744e9f730d4d0384f8c4b186ddc9f06cbd5f k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0],SizeBytes:9068367,},ContainerImage{Names:[k8s.gcr.io/sig-storage/csi-node-driver-registrar@sha256:f9bcee63734b7b01555ee8fc8fb01ac2922478b2c8934bf8d468dd2916edc405 k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0],SizeBytes:8582494,},ContainerImage{Names:[k8s.gcr.io/sig-storage/livenessprobe@sha256:529be2c9770add0cdd0c989115222ea9fc1be430c11095eb9f6dafcf98a36e2b k8s.gcr.io/sig-storage/livenessprobe:v2.4.0],SizeBytes:7960518,},ContainerImage{Names:[k8s.gcr.io/kas-network-proxy/proxy-agent@sha256:a9fbc49846c679f3d0f6fb2606ad3346fca0d915d7de3ee177f09551a1b1ec1b k8s.gcr.io/kas-network-proxy/proxy-agent:v0.0.30],SizeBytes:7711912,},ContainerImage{Names:[k8s.gcr.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a k8s.gcr.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:732746,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c k8s.gcr.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May 25 07:38:09.417: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-nndt
May 25 07:38:09.458: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-nndt
May 25 07:38:09.506: INFO: metrics-server-v0.5.2-fcb7b948c-mcrqn started at 2022-05-25 05:43:36 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:09.506: INFO: Container metrics-server ready: true, restart count 0
May 25 07:38:09.506: INFO: Container metrics-server-nanny ready: true, restart count 0
May 25 07:38:09.506: INFO: kube-proxy-bootstrap-e2e-minion-group-nndt started at 2022-05-25 05:43:05 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.506: INFO: Container kube-proxy ready: true, restart count 0
May 25 07:38:09.506: INFO: metadata-proxy-v0.1-smd54 started at 2022-05-25 05:43:06 +0000 UTC (0+2 container statuses recorded)
May 25 07:38:09.506: INFO: Container metadata-proxy ready: true, restart count 0
May 25 07:38:09.506: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
May 25 07:38:09.506: INFO: konnectivity-agent-jd9ml started at 2022-05-25 05:43:21 +0000 UTC (0+1 container statuses recorded)
May 25 07:38:09.506: INFO: Container konnectivity-agent ready: true, restart count 0
May 25 07:38:09.681: INFO:
Latency metrics for node bootstrap-e2e-minion-group-nndt
�[1mSTEP�[0m: Waiting for namespaces [snapshotting-5728-3780] to vanish
[AfterEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource]
test/e2e/framework/framework.go:187
May 25 07:38:11.795: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
from junit_01.xml
Filter through log files | View test history on testgrid
error during ./hack/ginkgo-e2e.sh --ginkgo.focus=\[Feature:VolumeSnapshotDataSource\] --ginkgo.skip=\[Disruptive\]|\[Flaky\] --minStartupPods=8 --report-dir=/workspace/_artifacts --disable-log-dump=true (interrupted): exit status 1
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest --timeout triggered
from junit_runner.xml
Filter through log files | View test history on testgrid
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable-stress[Feature:VolumeSnapshotDataSource] should support snapshotting of many volumes repeatedly [Slow] [Serial]
Kubernetes e2e suite [sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Ephemeral Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works, check deletion (ephemeral)
Kubernetes e2e suite [sig-storage] CSI mock volume CSI Volume Snapshots [Feature:VolumeSnapshotDataSource] volumesnapshotcontent and pvc in Bound state with deletion timestamp set should not get deleted while snapshot finalizer exists
kubetest Check APIReachability
kubetest Deferred TearDown
kubetest DumpClusterLogs
kubetest Extract
kubetest GetDeployer
kubetest IsUp
kubetest Prepare
kubetest TearDown
kubetest TearDown Previous
kubetest Up
kubetest diffResources
kubetest kubectl version
kubetest list nodes
kubetest listResources After
kubetest listResources Before
kubetest listResources Down
kubetest listResources Up
kubetest test setup
Kubernetes e2e suite [sig-api-machinery] API priority and fairness should ensure that requests can be classified by adding FlowSchema and PriorityLevelConfiguration
Kubernetes e2e suite [sig-api-machinery] API priority and fairness should ensure that requests can't be drowned out (priority)
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] listing mutating webhooks should work [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny attaching pod [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate configmap [Conformance]
Kubernetes e2e suite [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with pruning [Conformance]
Kubernetes e2e suite [sig-api-machinery] Aggregator Should be able to support the 1.17 Sample API Server using the current Aggregator [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourceDefinition Watch [Privileged:ClusterAdmin] CustomResourceDefinition Watch watch on custom resource definition objects [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] Simple CustomResourceDefinition getting/updating/patching custom resource definition status sub-resource works [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] Simple CustomResourceDefinition listing custom resource definition objects works [Conformance]
Kubernetes e2e suite [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD preserving unknown fields in an embedded object [Conformance]
Kubernetes e2e suite [sig-api-machinery] Discovery Custom resource should have storage version hash
Kubernetes e2e suite [sig-api-machinery] Etcd failure [Disruptive] should recover from SIGKILL
Kubernetes e2e suite [sig-api-machinery] Garbage collector should delete pods created by rc when not orphaning [Conformance]
Kubernetes e2e suite [sig-api-machinery] Garbage collector should keep the rc around until all its pods are deleted if the deleteOptions says so [Conformance]
Kubernetes e2e suite [sig-api-machinery] Garbage collector should not be blocked by dependency circle [Conformance]
Kubernetes e2e suite [sig-api-machinery] Garbage collector should not delete dependents that have both valid owner and owner that's waiting for dependents to be deleted [Conformance]
Kubernetes e2e suite [sig-api-machinery] Garbage collector should support cascading deletion of custom resources
Kubernetes e2e suite [sig-api-machinery] Generated clientset should create pods, set the deletionTimestamp and deletionGracePeriodSeconds of the pod
Kubernetes e2e suite [sig-api-machinery] Generated clientset should create v1 cronJobs, delete cronJobs, watch cronJobs
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's multiple priority class scope (quota set to pod count: 2) against 2 pods with same priority classes.
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's priority class scope (cpu, memory quota set) against a pod with same priority class.
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's priority class scope (quota set to pod count: 1) against 2 pods with different priority class.
Kubernetes e2e suite [sig-api-machinery] ResourceQuota [Feature:PodPriority] should verify ResourceQuota's priority class scope (quota set to pod count: 1) against a pod with different priority class (ScopeSelectorOpNotIn).
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a persistent volume claim
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a replica set. [Conformance]
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a secret. [Conformance]
Kubernetes e2e suite [sig-api-machinery] ResourceQuota should verify ResourceQuota with cross namespace pod affinity scope using scope-selectors.
Kubernetes e2e suite [sig-api-machinery] Server request timeout should return HTTP status code 400 if the user specifies an invalid timeout in the request URL
Kubernetes e2e suite [sig-api-machinery] ServerSideApply should give up ownership of a field if forced applied by a controller
Kubernetes e2e suite [sig-api-machinery] ServerSideApply should not remove a field if an owner unsets the field but other managers still have ownership of the field
Kubernetes e2e suite [sig-api-machinery] Servers with support for API chunking should support continue listing from the last key if the original version has been compacted away, though the list is inconsistent [Slow]
Kubernetes e2e suite [sig-api-machinery] Watchers should receive events on concurrent watches in same order [Conformance]
Kubernetes e2e suite [sig-api-machinery] client-go should negotiate watch and report errors with accept "application/json,application/vnd.kubernetes.protobuf"
Kubernetes e2e suite [sig-api-machinery] server version should find the server version [Conformance]
Kubernetes e2e suite [sig-apps] CronJob should be able to schedule after more than 100 missed schedule
Kubernetes e2e suite [sig-apps] DaemonRestart [Disruptive] Kube-proxy should recover after being killed accidentally
Kubernetes e2e suite [sig-apps] Deployment Deployment should have a working scale subresource [Conformance]
Kubernetes e2e suite [sig-apps] Deployment deployment should support proportional scaling [Conformance]
Kubernetes e2e suite [sig-apps] DisruptionController Listing PodDisruptionBudgets for all namespaces should list and delete a collection of PodDisruptionBudgets [Conformance]
Kubernetes e2e suite [sig-apps] DisruptionController evictions: no PDB => should allow an eviction
Kubernetes e2e suite [sig-apps] DisruptionController should block an eviction until the PDB is updated to allow it [Conformance]
Kubernetes e2e suite [sig-apps] Job should delete a job [Conformance]
Kubernetes e2e suite [sig-apps] Job should not create pods when created in suspend state
Kubernetes e2e suite [sig-apps] Job should run a job to completion when tasks sometimes fail and are not locally restarted
Kubernetes e2e suite [sig-apps] Job should run a job to completion when tasks succeed
Kubernetes e2e suite [sig-apps] ReplicaSet should adopt matching pods on creation and release no longer matching pods [Conformance]
Kubernetes e2e suite [sig-apps] ReplicaSet should surface a failure condition on a common issue like exceeded quota
Kubernetes e2e suite [sig-apps] ReplicationController should adopt matching pods on creation [Conformance]
Kubernetes e2e suite [sig-apps] ReplicationController should serve a basic image on each replica with a private image
Kubernetes e2e suite [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should adopt matching orphans and release non-matching pods
Kubernetes e2e suite [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should perform rolling updates and roll backs of template modifications [Conformance]
Kubernetes e2e suite [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] should provide basic identity
Kubernetes e2e suite [sig-apps] StatefulSet Deploy clustered applications [Feature:StatefulSet] [Slow] should creating a working redis cluster
Kubernetes e2e suite [sig-apps] StatefulSet Non-retain StatefulSetPersistentVolumeClaimPolicy [Feature:StatefulSetAutoDeletePVC] should delete PVCs with a OnScaledown policy
Kubernetes e2e suite [sig-apps] stateful Upgrade [Feature:StatefulUpgrade] stateful upgrade should maintain a functioning cluster
Kubernetes e2e suite [sig-auth] Certificates API [Privileged:ClusterAdmin] should support CSR API operations [Conformance]
Kubernetes e2e suite [sig-auth] ServiceAccounts no secret-based service account token should be auto-generated
Kubernetes e2e suite [sig-auth] ServiceAccounts should guarantee kube-root-ca.crt exist in any namespace [Conformance]
Kubernetes e2e suite [sig-auth] ServiceAccounts should support InClusterConfig with token rotation [Slow]
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthenticator] The kubelet can delegate ServiceAccount tokens to the API server
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthorizer] A node shouldn't be able to delete another node
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthorizer] Getting a non-existent configmap should exit with the Forbidden error, not a NotFound error
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthorizer] Getting a non-existent secret should exit with the Forbidden error, not a NotFound error
Kubernetes e2e suite [sig-auth] [Feature:NodeAuthorizer] Getting an existing configmap should exit with the Forbidden error
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaler scalability [Slow] CA ignores unschedulable pods while scheduling schedulable pods [Feature:ClusterAutoscalerScalability6]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaler scalability [Slow] should scale up twice [Feature:ClusterAutoscalerScalability2]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaler scalability [Slow] shouldn't scale down with underutilized nodes due to host port conflicts [Feature:ClusterAutoscalerScalability5]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] Should be able to scale a node group up from 0[Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] Should scale down GPU pool from 1 [GpuType:] [Feature:ClusterSizeAutoscalingGpu]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] should add node to the particular mig [Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] shouldn't be able to scale down when rescheduling a pod is required, but pdb doesn't allow drain[Feature:ClusterSizeAutoscalingScaleDown]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] shouldn't increase cluster size if pending pod is too large [Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] Cluster size autoscaling [Slow] shouldn't trigger additional scale-ups during processing scale-up [Feature:ClusterSizeAutoscalingScaleUp]
Kubernetes e2e suite [sig-autoscaling] [Feature:ClusterSizeAutoscalingScaleUp] [Slow] Autoscaling Autoscaling a service from 1 pod and 3 nodes to 8 pods and >=4 nodes takes less than 15 minutes
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) ReplicationController light Should scale from 2 pods to 1 pod [Slow]
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) [Serial] [Slow] Deployment Should scale from 1 pod to 3 pods and from 3 to 5
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) [Serial] [Slow] ReplicaSet Should scale from 1 pod to 3 pods and from 3 to 5
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) [Serial] [Slow] ReplicaSet with idle sidecar (ContainerResource use case) Should not scale up on a busy sidecar with an idle application
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) [Serial] [Slow] ReplicationController Should scale from 1 pod to 3 pods and from 3 to 5 and verify decision stability
Kubernetes e2e suite [sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) [Serial] [Slow] ReplicationController Should scale from 5 pods to 3 pods and from 3 to 1 and verify decision stability
Kubernetes e2e suite [sig-autoscaling] [HPA] Horizontal pod autoscaling (scale resource: Custom Metrics from Stackdriver) should scale down with Custom Metric of type Pod from Stackdriver [Feature:CustomMetricsAutoscaling]
Kubernetes e2e suite [sig-cli] Kubectl Port forwarding With a server listening on 0.0.0.0 that expects a client request should support a client that connects, sends DATA, and disconnects
Kubernetes e2e suite [sig-cli] Kubectl Port forwarding With a server listening on 0.0.0.0 that expects a client request should support a client that connects, sends NO DATA, and disconnects
Kubernetes e2e suite [sig-cli] Kubectl Port forwarding With a server listening on localhost that expects NO client request should support a client that connects, sends DATA, and disconnects
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl api-versions should check if v1 is in available api versions [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl apply apply set/view last-applied
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl create quota should create a quota with scopes
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl create quota should reject quota with invalid scopes
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl diff should check if kubectl diff finds a difference for Deployments [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl get componentstatuses should get componentstatuses
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl logs should be able to retrieve and filter logs [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl validation should create/apply a valid CR with arbitrary-extra properties for CRD with partially-specified validation schema
Kubernetes e2e suite [sig-cli] Kubectl client Kubectl validation should detect unknown metadata fields in both the root and embedded object of a CR
Kubernetes e2e suite [sig-cli] Kubectl client Proxy server should support --unix-socket=/path [Conformance]
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should contain last line of the log
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should handle in-cluster config
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should return command exit codes [Slow] running a failing command with --leave-stdin-open
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should return command exit codes [Slow] running a failing command without --restart=Never
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should return command exit codes [Slow] running a failing command without --restart=Never, but with --rm
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should return command exit codes execing into a container with a failing command
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should return command exit codes execing into a container with a successful command
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should support exec through kubectl proxy
Kubernetes e2e suite [sig-cli] Kubectl client Simple pod should support inline execution and attach
Kubernetes e2e suite [sig-cloud-provider-gcp] GKE node pools [Feature:GKENodePool] should create a cluster with multiple node pools [Feature:GKENodePool]
Kubernetes e2e suite [sig-cloud-provider-gcp] HA-master [Feature:HAMaster] survive addition/removal replicas multizone workers [Serial][Disruptive]
Kubernetes e2e suite [sig-cloud-provider-gcp] Nodes [Disruptive] Resize [Slow] should be able to add nodes
Kubernetes e2e suite [sig-cloud-provider-gcp] Ports Security Check [Feature:KubeletSecurity] should not have port 10255 open on its all public IP addresses
Kubernetes e2e suite [sig-cloud-provider-gcp] Ports Security Check [Feature:KubeletSecurity] should not have port 4194 open on its all public IP addresses
Kubernetes e2e suite [sig-cloud-provider-gcp] Reboot [Disruptive] [Feature:Reboot] each node by ordering clean reboot and ensure they function upon restart
Kubernetes e2e suite [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade]
Kubernetes e2e suite [sig-cloud-provider-gcp] [Disruptive]NodeLease NodeLease deletion node lease should be deleted when corresponding node is deleted
Kubernetes e2e suite [sig-cluster-lifecycle] [Feature:BootstrapTokens] should delete the token secret when the secret expired
Kubernetes e2e suite [sig-cluster-lifecycle] [Feature:BootstrapTokens] should not delete the token secret when the secret is not expired
Kubernetes e2e suite [sig-instrumentation] Events API should delete a collection of events [Conformance]
Kubernetes e2e suite [sig-instrumentation] Events should ensure that an event can be fetched, patched, deleted, and listed [Conformance]
Kubernetes e2e suite [sig-instrumentation] Logging soak [Performance] [Slow] [Disruptive] should survive logging 1KB every 1s seconds, for a duration of 2m0s
Kubernetes e2e suite [sig-instrumentation] MetricsGrabber should grab all metrics from a Kubelet.
Kubernetes e2e suite [sig-network] Conntrack should be able to preserve UDP traffic when initial unready endpoints get ready
Kubernetes e2e suite [sig-network] Conntrack should be able to preserve UDP traffic when server pod cycles for a NodePort service
Kubernetes e2e suite [sig-network] DNS should provide DNS for the cluster [Conformance]
Kubernetes e2e suite [sig-network] DNS should resolve DNS of partial qualified names for the cluster [LinuxOnly]
Kubernetes e2e suite [sig-network] EndpointSlice should create and delete Endpoints and EndpointSlices for a Service with a selector specified [Conformance]
Kubernetes e2e suite [sig-network] EndpointSlice should have Endpoints and EndpointSlices pointing to API Server [Conformance]
Kubernetes e2e suite [sig-network] Firewall rule control plane should not expose well-known ports
Kubernetes e2e suite [sig-network] IngressClass [Feature:Ingress] should allow IngressClass to have Namespace-scoped parameters [Serial]
Kubernetes e2e suite [sig-network] IngressClass [Feature:Ingress] should set default value on new IngressClass [Serial]
Kubernetes e2e suite [sig-network] LoadBalancers should be able to create LoadBalancer Service without NodePort and change it [Slow]
Kubernetes e2e suite [sig-network] Loadbalancing: L7 GCE [Slow] [Feature:Ingress] should conform to Ingress spec
Kubernetes e2e suite [sig-network] Loadbalancing: L7 GCE [Slow] [Feature:NEG] [Flaky] should sync endpoints for both Ingress-referenced NEG and standalone NEG
Kubernetes e2e suite [sig-network] Netpol API should support creating NetworkPolicy API operations
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should allow egress access on one named port [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should allow ingress access from updated pod [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should deny ingress from pods on other namespaces [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce ingress policy allowing any port traffic to a server on a specific protocol [Feature:NetworkPolicy] [Feature:UDP]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy based on PodSelector and NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy based on PodSelector with MatchExpressions[Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce policy to allow ingress traffic from pods in all namespaces [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should enforce updated policy [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should not allow access by TCP when a policy specifies only UDP [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should stop enforcing policies after they are deleted [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Netpol NetworkPolicy between server and client should support denying of egress traffic on the client side (even if the server explicitly allows this traffic) [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicy API should support creating NetworkPolicy API operations
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should allow egress access to server in CIDR block [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should allow ingress access from namespace on one named port [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should allow ingress access from updated pod [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce multiple ingress policies with ingress allow-all policy taking precedence [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce policies to check ingress and egress policies can be controlled independently based on PodSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce policy to allow traffic only from a pod in a different namespace based on PodSelector and NamespaceSelector [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should enforce updated policy [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] NetworkPolicyLegacy [LinuxOnly] NetworkPolicy between server and client should support allow-all policy [Feature:NetworkPolicy]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Pods should function for intra-pod communication: http [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Pods should function for node-pod communication: http [LinuxOnly] [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Pods should function for node-pod communication: sctp [LinuxOnly][Feature:SCTPConnectivity][Disruptive]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Pods should function for node-pod communication: udp [LinuxOnly] [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for client IP based session affinity: udp [LinuxOnly]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for endpoint-Service: sctp [Feature:SCTPConnectivity][Disruptive]
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for endpoint-Service: udp
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for node-Service: udp
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should function for pod-Service: http
Kubernetes e2e suite [sig-network] Networking Granular Checks: Services should support basic nodePort: udp functionality
Kubernetes e2e suite [sig-network] Networking should provide Internet connection for containers [Feature:Networking-IPv4]
Kubernetes e2e suite [sig-network] Networking should provide Internet connection for containers [Feature:Networking-IPv6][Experimental][LinuxOnly]
Kubernetes e2e suite [sig-network] Proxy version v1 A set of valid responses are returned for both pod and service ProxyWithPath [Conformance]
Kubernetes e2e suite [sig-network] SCTP [LinuxOnly] should create a Pod with SCTP HostPort
Kubernetes e2e suite [sig-network] Service endpoints latency should not be very high [Conformance]
Kubernetes e2e suite [sig-network] Services should be able to change the type from ClusterIP to ExternalName [Conformance]
Kubernetes e2e suite [sig-network] Services should be able to change the type from ExternalName to NodePort [Conformance]
Kubernetes e2e suite [sig-network] Services should be able to change the type from NodePort to ExternalName [Conformance]
Kubernetes e2e suite [sig-network] Services should be able to up and down services
Kubernetes e2e suite [sig-network] Services should be able to update service type to NodePort listening on same port number but different protocols
Kubernetes e2e suite [sig-network] Services should be rejected when no endpoints exist
Kubernetes e2e suite [sig-network] Services should fallback to terminating endpoints when there are no ready endpoints with externallTrafficPolicy=Cluster [Feature:ProxyTerminatingEndpoints]
Kubernetes e2e suite [sig-network] Services should fallback to terminating endpoints when there are no ready endpoints with internalTrafficPolicy=Cluster [Feature:ProxyTerminatingEndpoints]
Kubernetes e2e suite [sig-network] Services should have session affinity timeout work for service with type clusterIP [LinuxOnly] [Conformance]
Kubernetes e2e suite [sig-network] Services should have session affinity work for NodePort service [LinuxOnly] [Conformance]
Kubernetes e2e suite [sig-network] Services should have session affinity work for service with type clusterIP [LinuxOnly] [Conformance]
Kubernetes e2e suite [sig-network] Services should implement service.kubernetes.io/headless
Kubernetes e2e suite [sig-network] Services should implement service.kubernetes.io/service-proxy-name
Kubernetes e2e suite [sig-network] Services should preserve source pod IP for traffic thru service cluster IP [LinuxOnly]
Kubernetes e2e suite [sig-network] Services should release NodePorts on delete
Kubernetes e2e suite [sig-network] Services should respect internalTrafficPolicy=Local Pod to Pod [Feature:ServiceInternalTrafficPolicy]
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] Granular Checks: Services Secondary IP Family [LinuxOnly] should function for pod-Service: http
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] Granular Checks: Services Secondary IP Family [LinuxOnly] should function for pod-Service: sctp [Feature:SCTPConnectivity][Disruptive]
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] Granular Checks: Services Secondary IP Family [LinuxOnly] should update endpoints: http
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] Granular Checks: Services Secondary IP Family [LinuxOnly] should update endpoints: udp
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] should create service with ipv4,v6 cluster ip
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] should create service with ipv6 cluster ip
Kubernetes e2e suite [sig-network] [Feature:IPv6DualStack] should create service with ipv6,v4 cluster ip
Kubernetes e2e suite [sig-network] [Feature:Topology Hints] should distribute endpoints evenly
Kubernetes e2e suite [sig-node] ConfigMap should be consumable via environment variable [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] ConfigMap should fail to create ConfigMap with empty key [Conformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test on terminated container should report termination message as empty when pod succeeds and TerminationMessagePolicy FallbackToLogsOnError is set [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test on terminated container should report termination message from log output if TerminationMessagePolicy FallbackToLogsOnError is set [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test on terminated container should report termination message if TerminationMessagePath is set [NodeConformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test on terminated container should report termination message if TerminationMessagePath is set as non-root user and at a non-default path [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test when running a container with a new image should be able to pull from private registry with secret [NodeConformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test when running a container with a new image should not be able to pull image from invalid registry [NodeConformance]
Kubernetes e2e suite [sig-node] Container Runtime blackbox test when starting a container that exits should run with the expected status [NodeConformance] [Conformance]
Kubernetes e2e suite [sig-nod