ci-cri-containerd-e2e-cos-gce-slow #1596480384315428864 Results

Test Failures

Kubernetes e2e suite [It] [sig-apps] CronJob should not schedule new jobs when ForbidConcurrent [Slow] [Conformance] 2m0s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sCronJob\sshould\snot\sschedule\snew\sjobs\swhen\sForbidConcurrent\s\[Slow\]\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0004bf860)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-apps] CronJob
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:52:07.623
Nov 26 12:52:07.623: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename cronjob 11/26/22 12:52:07.625
Nov 26 12:54:07.669: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc004bb2840>: {
        msg: "wait for service account \"default\" in namespace \"cronjob-1038\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc0001c1a00>{
            s: "timed out waiting for the condition",
        },
    }
Nov 26 12:54:07.670: FAIL: wait for service account "default" in namespace "cronjob-1038": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0004bf860)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-apps] CronJob
  test/e2e/framework/node/init/init.go:32
Nov 26 12:54:07.670: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] CronJob
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:54:07.713
STEP: Collecting events from namespace "cronjob-1038". 11/26/22 12:54:07.713
STEP: Found 0 events. 11/26/22 12:54:07.753
Nov 26 12:54:07.794: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 26 12:54:07.794: INFO: 
Nov 26 12:54:07.836: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:54:07.878: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 11767 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:52:08 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:54:07.879: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:54:07.922: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:54:07.965: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 26 12:54:07.965: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:54:08.015: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 12080 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-5002":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-1783":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-6237":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-7979":"bootstrap-e2e-minion-group-8pmq","csi-mock-csi-mock-volumes-3803":"bootstrap-e2e-minion-group-8pmq"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:48:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-26 12:51:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:53:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f7c6a89c-6d87-11ed-8904-52ce12031d16,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f6ec6f89-6d87-11ed-8904-52ce12031d16,DevicePath:,},},Config:nil,},}
Nov 26 12:54:08.015: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:54:08.064: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:54:08.107: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-8pmq: error trying to reach service: No agent available
Nov 26 12:54:08.107: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:54:08.149: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 12110 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-5987":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-1645":"bootstrap-e2e-minion-group-dldd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:43:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-26 12:51:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:54:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:54:06 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:54:06 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:54:06 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:54:06 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:54:08.150: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:54:08.193: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:54:08.238: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-dldd: error trying to reach service: No agent available
Nov 26 12:54:08.238: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:54:08.298: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 12023 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volumemode-3323":"bootstrap-e2e-minion-group-tvmd","csi-mock-csi-mock-volumes-9921":"bootstrap-e2e-minion-group-tvmd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-26 12:48:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-26 12:51:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:53:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:53:30 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:53:30 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:53:30 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:53:30 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},},Config:nil,},}
Nov 26 12:54:08.298: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:54:08.344: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:54:08.417: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-tvmd: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-apps] CronJob
  tear down framework | framework.go:193
STEP: Destroying namespace "cronjob-1038" for this suite. 11/26/22 12:54:08.417

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] 7m28s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sStatefulSet\sBasic\sStatefulSet\sfunctionality\s\[StatefulSetBasic\]\sBurst\sscaling\sshould\srun\sto\scompletion\seven\swith\sunhealthy\spods\s\[Slow\]\s\[Conformance\]$'

test/e2e/framework/statefulset/wait.go:120
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
	test/e2e/framework/statefulset/wait.go:120 +0x231
k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
	test/e2e/apps/statefulset.go:723 +0x425
There were additional failures detected after the initial failure:
[FAILED]
Nov 26 12:40:52.292: Get "https://34.145.15.180/apis/apps/v1/namespaces/statefulset-1696/statefulsets": dial tcp 34.145.15.180:443: connect: connection refused
In [AfterEach] at: test/e2e/framework/statefulset/rest.go:76
----------
[FAILED]
Nov 26 12:40:52.371: failed to list events in namespace "statefulset-1696": Get "https://34.145.15.180/api/v1/namespaces/statefulset-1696/events": dial tcp 34.145.15.180:443: connect: connection refused
In [DeferCleanup (Each)] at: test/e2e/framework/debug/dump.go:44
----------
[FAILED]
Nov 26 12:40:52.412: Couldn't delete ns: "statefulset-1696": Delete "https://34.145.15.180/api/v1/namespaces/statefulset-1696": dial tcp 34.145.15.180:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.145.15.180/api/v1/namespaces/statefulset-1696", Err:(*net.OpError)(0xc00065d400)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370

				[BeforeEach] [sig-apps] StatefulSet
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:33:23.677
Nov 26 12:33:23.678: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename statefulset 11/26/22 12:33:23.679
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:33:23.938
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:33:24.109
[BeforeEach] [sig-apps] StatefulSet
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-apps] StatefulSet
  test/e2e/apps/statefulset.go:98
[BeforeEach] Basic StatefulSet functionality [StatefulSetBasic]
  test/e2e/apps/statefulset.go:113
STEP: Creating service test in namespace statefulset-1696 11/26/22 12:33:24.219
[It] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance]
  test/e2e/apps/statefulset.go:697
STEP: Creating stateful set ss in namespace statefulset-1696 11/26/22 12:33:24.327
STEP: Waiting until all stateful set ss replicas will be running in namespace statefulset-1696 11/26/22 12:33:24.444
Nov 26 12:33:24.533: INFO: Found 0 stateful pods, waiting for 1
Nov 26 12:33:34.591: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false
Nov 26 12:33:44.575: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false
Nov 26 12:33:54.575: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
STEP: Confirming that stateful set scale up will not halt with unhealthy stateful pod 11/26/22 12:33:54.576
Nov 26 12:33:54.617: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:33:55.194: INFO: stderr: "+ mv -v /usr/local/apache2/htdocs/index.html /tmp/\n"
Nov 26 12:33:55.194: INFO: stdout: "'/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'\n"
Nov 26 12:33:55.194: INFO: stdout of mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true on ss-0: '/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'

Nov 26 12:33:55.236: INFO: Waiting for pod ss-0 to enter Running - Ready=false, currently Running - Ready=true
Nov 26 12:34:05.279: INFO: Waiting for pod ss-0 to enter Running - Ready=false, currently Running - Ready=false
Nov 26 12:34:05.279: INFO: Waiting for statefulset status.replicas updated to 0
Nov 26 12:34:05.456: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:05.456: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:05.456: INFO: 
Nov 26 12:34:05.456: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:06.499: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:06.499: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:06.499: INFO: 
Nov 26 12:34:06.499: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:07.540: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:07.541: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:07.541: INFO: 
Nov 26 12:34:07.541: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:08.582: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:08.582: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:08.582: INFO: 
Nov 26 12:34:08.582: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:09.624: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:09.624: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:09.624: INFO: 
Nov 26 12:34:09.624: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:10.667: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:10.667: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:10.667: INFO: 
Nov 26 12:34:10.667: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:11.713: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:11.713: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:11.713: INFO: 
Nov 26 12:34:11.713: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:12.757: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:12.757: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:12.757: INFO: 
Nov 26 12:34:12.757: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:13.800: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:13.800: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:13.800: INFO: 
Nov 26 12:34:13.800: INFO: StatefulSet ss has not reached scale 3, at 1
Nov 26 12:34:14.867: INFO: POD   NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:34:14.867: INFO: ss-0  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:55 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:33:31 +0000 UTC  }]
Nov 26 12:34:14.867: INFO: 
Nov 26 12:34:14.867: INFO: StatefulSet ss has not reached scale 3, at 1
STEP: Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-1696 11/26/22 12:34:15.867
Nov 26 12:34:15.909: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:34:16.427: INFO: stderr: "+ mv -v /tmp/index.html /usr/local/apache2/htdocs/\n"
Nov 26 12:34:16.427: INFO: stdout: "'/tmp/index.html' -> '/usr/local/apache2/htdocs/index.html'\n"
Nov 26 12:34:16.427: INFO: stdout of mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true on ss-0: '/tmp/index.html' -> '/usr/local/apache2/htdocs/index.html'

Nov 26 12:34:16.480: INFO: Found 1 stateful pods, waiting for 3
Nov 26 12:34:26.540: INFO: Found 1 stateful pods, waiting for 3
Nov 26 12:34:36.538: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:34:36.538: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Pending - Ready=false
Nov 26 12:34:46.555: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:34:46.555: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Pending - Ready=false
Nov 26 12:34:56.545: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:34:56.545: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Running - Ready=false
Nov 26 12:35:06.565: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:35:06.565: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Running - Ready=false
Nov 26 12:35:16.532: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:35:16.532: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:35:16.532: INFO: Waiting for pod ss-2 to enter Running - Ready=true, currently Running - Ready=true
STEP: Scale down will not halt with unhealthy stateful pod 11/26/22 12:35:16.532
Nov 26 12:35:16.599: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:35:17.181: INFO: rc: 1
Nov 26 12:35:17.181: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:35:27.182: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:35:27.621: INFO: rc: 1
Nov 26 12:35:27.621: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:35:37.622: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:35:38.023: INFO: rc: 1
Nov 26 12:35:38.023: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:35:48.024: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:36:10.944: INFO: stderr: "+ mv -v /usr/local/apache2/htdocs/index.html /tmp/\n"
Nov 26 12:36:10.944: INFO: stdout: "'/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'\n"
Nov 26 12:36:10.944: INFO: stdout of mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true on ss-0: '/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'

Nov 26 12:36:10.944: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-1 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:36:11.538: INFO: stderr: "+ mv -v /usr/local/apache2/htdocs/index.html /tmp/\n"
Nov 26 12:36:11.538: INFO: stdout: "'/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'\n"
Nov 26 12:36:11.538: INFO: stdout of mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true on ss-1: '/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'

Nov 26 12:36:11.538: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-1696 exec ss-2 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:36:12.132: INFO: stderr: "+ mv -v /usr/local/apache2/htdocs/index.html /tmp/\n"
Nov 26 12:36:12.132: INFO: stdout: "'/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'\n"
Nov 26 12:36:12.132: INFO: stdout of mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true on ss-2: '/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'

Nov 26 12:36:12.132: INFO: Waiting for statefulset status.replicas updated to 0
Nov 26 12:36:12.173: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 3
Nov 26 12:36:22.217: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 3
Nov 26 12:36:32.214: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 3
Nov 26 12:36:42.215: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 3
Nov 26 12:36:52.219: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 3
Nov 26 12:37:02.227: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:37:12.239: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:37:22.229: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:37:32.229: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:37:42.250: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:37:52.214: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:38:02.214: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:38:12.214: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:38:22.214: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 5m0.65s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 5m0s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 3m7.795s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:38:32.216: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:38:42.214: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 5m20.652s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 5m20.002s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 3m27.797s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:38:52.221: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:39:02.213: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 5m40.655s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 5m40.005s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 3m47.8s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:39:12.236: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 2
Nov 26 12:39:22.251: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 6m0.657s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 6m0.008s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 4m7.803s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:39:32.226: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
Nov 26 12:39:42.347: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 6m20.66s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 6m20.01s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 4m27.805s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:39:52.225: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
Nov 26 12:40:02.230: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 6m40.667s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 6m40.017s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 4m47.812s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:40:12.300: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
Nov 26 12:40:22.268: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 7m0.669s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 7m0.019s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 5m7.814s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:40:32.259: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
Nov 26 12:40:42.229: INFO: Waiting for stateful set status.readyReplicas to become 0, currently 1
------------------------------
Progress Report for Ginkgo Process #19
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Burst scaling should run to completion even with unhealthy pods [Slow] [Conformance] (Spec Runtime: 7m20.671s)
    test/e2e/apps/statefulset.go:697
    In [It] (Node Runtime: 7m20.021s)
      test/e2e/apps/statefulset.go:697
      At [By Step] Scale down will not halt with unhealthy stateful pod (Step Runtime: 5m27.816s)
        test/e2e/apps/statefulset.go:721

      Spec Goroutine
      goroutine 841 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc00252e2d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x40?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc003ba5de8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76fb525?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
          test/e2e/framework/statefulset/wait.go:104
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
          test/e2e/apps/statefulset.go:723
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004ee4000})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:40:52.213: FAIL: Failed waiting for stateful set status.readyReplicas updated to 0: Get "https://34.145.15.180/apis/apps/v1/namespaces/statefulset-1696/statefulsets/ss": dial tcp 34.145.15.180:443: connect: connection refused

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForStatusReadyReplicas({0x801de88?, 0xc001c13d40}, 0xc0052c2500, 0x0)
	test/e2e/framework/statefulset/wait.go:120 +0x231
k8s.io/kubernetes/test/e2e/apps.glob..func10.2.11()
	test/e2e/apps/statefulset.go:723 +0x425
[AfterEach] Basic StatefulSet functionality [StatefulSetBasic]
  test/e2e/apps/statefulset.go:124
Nov 26 12:40:52.252: INFO: Deleting all statefulset in ns statefulset-1696
Nov 26 12:40:52.292: INFO: Unexpected error: 
    <*url.Error | 0xc004e88480>: {
        Op: "Get",
        URL: "https://34.145.15.180/apis/apps/v1/namespaces/statefulset-1696/statefulsets",
        Err: <*net.OpError | 0xc004032370>{
            Op: "dial",
            Net: "tcp",
            Source: nil,
            Addr: <*net.TCPAddr | 0xc001bb3380>{
                IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 34, 145, 15, 180],
                Port: 443,
                Zone: "",
            },
            Err: <*os.SyscallError | 0xc000fe42c0>{
                Syscall: "connect",
                Err: <syscall.Errno>0x6f,
            },
        },
    }
Nov 26 12:40:52.292: FAIL: Get "https://34.145.15.180/apis/apps/v1/namespaces/statefulset-1696/statefulsets": dial tcp 34.145.15.180:443: connect: connection refused

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/statefulset.DeleteAllStatefulSets({0x801de88, 0xc001c13d40}, {0xc005280e00, 0x10})
	test/e2e/framework/statefulset/rest.go:76 +0x113
k8s.io/kubernetes/test/e2e/apps.glob..func10.2.2()
	test/e2e/apps/statefulset.go:129 +0x1b2
[AfterEach] [sig-apps] StatefulSet
  test/e2e/framework/node/init/init.go:32
Nov 26 12:40:52.292: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] StatefulSet
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-apps] StatefulSet
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:40:52.331
STEP: Collecting events from namespace "statefulset-1696". 11/26/22 12:40:52.332
Nov 26 12:40:52.371: INFO: Unexpected error: failed to list events in namespace "statefulset-1696": 
    <*url.Error | 0xc004e88900>: {
        Op: "Get",
        URL: "https://34.145.15.180/api/v1/namespaces/statefulset-1696/events",
        Err: <*net.OpError | 0xc004032730>{
            Op: "dial",
            Net: "tcp",
            Source: nil,
            Addr: <*net.TCPAddr | 0xc001bb38f0>{
                IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 34, 145, 15, 180],
                Port: 443,
                Zone: "",
            },
            Err: <*os.SyscallError | 0xc000fe4620>{
                Syscall: "connect",
                Err: <syscall.Errno>0x6f,
            },
        },
    }
Nov 26 12:40:52.371: FAIL: failed to list events in namespace "statefulset-1696": Get "https://34.145.15.180/api/v1/namespaces/statefulset-1696/events": dial tcp 34.145.15.180:443: connect: connection refused

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/debug.dumpEventsInNamespace(0xc0018e65c0, {0xc005280e00, 0x10})
	test/e2e/framework/debug/dump.go:44 +0x191
k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc001c13d40}, {0xc005280e00, 0x10})
	test/e2e/framework/debug/dump.go:62 +0x8d
k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc0018e6650?, {0xc005280e00?, 0x7fa7740?})
	test/e2e/framework/debug/init/init.go:34 +0x32
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
	test/e2e/framework/framework.go:274 +0x6d
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0011041e0)
	test/e2e/framework/framework.go:271 +0x179
reflect.Value.call({0x6627cc0?, 0xc004c9bdf0?, 0xc00104efb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0xc001f97a88?})
	/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc004c9bdf0?, 0x29449fc?}, {0xae73300?, 0xc00104ef80?, 0x0?})
	/usr/local/go/src/reflect/value.go:368 +0xbc
[DeferCleanup (Each)] [sig-apps] StatefulSet
  tear down framework | framework.go:193
STEP: Destroying namespace "statefulset-1696" for this suite. 11/26/22 12:40:52.372
Nov 26 12:40:52.412: FAIL: Couldn't delete ns: "statefulset-1696": Delete "https://34.145.15.180/api/v1/namespaces/statefulset-1696": dial tcp 34.145.15.180:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.145.15.180/api/v1/namespaces/statefulset-1696", Err:(*net.OpError)(0xc00065d400)})

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
	test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc0011041e0)
	test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc004c9bd70?, 0xc003ac0fb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
	/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc004c9bd70?, 0x0?}, {0xae73300?, 0x5?, 0xc000db63a8?})
	/usr/local/go/src/reflect/value.go:368 +0xbc

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] 22m53s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sStatefulSet\sBasic\sStatefulSet\sfunctionality\s\[StatefulSetBasic\]\sScaling\sshould\shappen\sin\spredictable\sorder\sand\shalt\sif\sany\sstateful\spod\sis\sunhealthy\s\[Slow\]\s\[Conformance\]$'

test/e2e/framework/statefulset/wait.go:58
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
	test/e2e/framework/statefulset/wait.go:58 +0xf9
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
	test/e2e/framework/statefulset/wait.go:80
k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
	test/e2e/apps/statefulset.go:643 +0x6d0
				[BeforeEach] [sig-apps] StatefulSet
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:40:50.688
Nov 26 12:40:50.689: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename statefulset 11/26/22 12:40:50.69
Nov 26 12:40:50.729: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:52.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:54.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:56.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:58.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:00.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:02.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:04.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:06.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:08.770: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:10.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:12.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:14.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:16.769: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:42:52.95
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:42:53.035
[BeforeEach] [sig-apps] StatefulSet
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-apps] StatefulSet
  test/e2e/apps/statefulset.go:98
[BeforeEach] Basic StatefulSet functionality [StatefulSetBasic]
  test/e2e/apps/statefulset.go:113
STEP: Creating service test in namespace statefulset-5414 11/26/22 12:42:56.894
[It] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance]
  test/e2e/apps/statefulset.go:587
STEP: Initializing watcher for selector baz=blah,foo=bar 11/26/22 12:42:56.937
STEP: Creating stateful set ss in namespace statefulset-5414 11/26/22 12:42:56.978
STEP: Waiting until all stateful set ss replicas will be running in namespace statefulset-5414 11/26/22 12:42:57.024
Nov 26 12:42:57.065: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Pending - Ready=false
Nov 26 12:43:07.127: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
STEP: Confirming that stateful set scale up will halt with unhealthy stateful pod 11/26/22 12:43:07.127
Nov 26 12:43:07.179: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:43:07.586: INFO: rc: 1
Nov 26 12:43:07.586: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:43:17.586: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:43:18.015: INFO: rc: 1
Nov 26 12:43:18.015: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:43:28.016: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:43:28.428: INFO: rc: 1
Nov 26 12:43:28.428: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:43:38.429: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:43:38.922: INFO: rc: 1
Nov 26 12:43:38.922: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:43:48.923: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:43:49.325: INFO: rc: 1
Nov 26 12:43:49.325: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:43:59.326: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:43:59.742: INFO: rc: 1
Nov 26 12:43:59.742: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:44:09.743: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:44:10.185: INFO: rc: 1
Nov 26 12:44:10.185: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:44:20.186: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:44:20.667: INFO: rc: 1
Nov 26 12:44:20.667: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:44:30.668: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:44:31.105: INFO: rc: 1
Nov 26 12:44:31.105: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:44:41.105: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:44:41.548: INFO: rc: 1
Nov 26 12:44:41.548: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:44:51.549: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:44:51.910: INFO: rc: 1
Nov 26 12:44:51.910: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:45:01.911: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:45:02.244: INFO: rc: 1
Nov 26 12:45:02.244: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:45:12.245: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:45:12.593: INFO: rc: 1
Nov 26 12:45:12.593: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
Nov 26 12:45:22.594: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:45:22.961: INFO: rc: 1
Nov 26 12:45:22.961: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
Nov 26 12:45:32.962: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:45:33.325: INFO: rc: 1
Nov 26 12:45:33.325: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
Nov 26 12:45:43.326: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:45:43.687: INFO: rc: 1
Nov 26 12:45:43.687: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
Nov 26 12:45:53.688: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:45:54.041: INFO: rc: 1
Nov 26 12:45:54.041: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
Nov 26 12:46:04.041: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:46:04.473: INFO: rc: 1
Nov 26 12:46:04.473: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
Nov 26 12:46:14.473: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true'
Nov 26 12:46:15.104: INFO: stderr: "+ mv -v /usr/local/apache2/htdocs/index.html /tmp/\n"
Nov 26 12:46:15.104: INFO: stdout: "'/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'\n"
Nov 26 12:46:15.104: INFO: stdout of mv -v /usr/local/apache2/htdocs/index.html /tmp/ || true on ss-0: '/usr/local/apache2/htdocs/index.html' -> '/tmp/index.html'

Nov 26 12:46:15.149: INFO: Waiting for pod ss-0 to enter Running - Ready=false, currently Running - Ready=true
Nov 26 12:46:25.194: INFO: Waiting for pod ss-0 to enter Running - Ready=false, currently Running - Ready=false
Nov 26 12:46:25.194: INFO: Waiting for statefulset status.replicas updated to 0
Nov 26 12:46:25.367: INFO: Verifying statefulset ss doesn't scale past 1 for another 9.999999034s
Nov 26 12:46:26.410: INFO: Verifying statefulset ss doesn't scale past 1 for another 8.955754836s
Nov 26 12:46:27.453: INFO: Verifying statefulset ss doesn't scale past 1 for another 7.912579803s
Nov 26 12:46:28.496: INFO: Verifying statefulset ss doesn't scale past 1 for another 6.869076761s
Nov 26 12:46:29.539: INFO: Verifying statefulset ss doesn't scale past 1 for another 5.82666115s
Nov 26 12:46:30.582: INFO: Verifying statefulset ss doesn't scale past 1 for another 4.783456986s
Nov 26 12:46:31.626: INFO: Verifying statefulset ss doesn't scale past 1 for another 3.740134397s
Nov 26 12:46:32.669: INFO: Verifying statefulset ss doesn't scale past 1 for another 2.697017851s
Nov 26 12:46:33.711: INFO: Verifying statefulset ss doesn't scale past 1 for another 1.654407612s
Nov 26 12:46:34.755: INFO: Verifying statefulset ss doesn't scale past 1 for another 611.481081ms
STEP: Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 11/26/22 12:46:35.755
Nov 26 12:46:35.798: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:46:36.146: INFO: rc: 1
Nov 26 12:46:36.146: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:46:46.147: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:46:46.480: INFO: rc: 1
Nov 26 12:46:46.480: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:46:56.481: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:46:56.815: INFO: rc: 1
Nov 26 12:46:56.815: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:47:06.816: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:47:07.144: INFO: rc: 1
Nov 26 12:47:07.144: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:47:17.145: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:47:17.479: INFO: rc: 1
Nov 26 12:47:17.479: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:47:27.480: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:47:27.815: INFO: rc: 1
Nov 26 12:47:27.815: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:47:37.816: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:47:38.148: INFO: rc: 1
Nov 26 12:47:38.148: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
Nov 26 12:47:48.148: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 7m6.25s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 5m0.001s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 1m21.183s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.ExecWithFullOutput({0xc002f24000?, 0x0?})
          test/e2e/framework/kubectl/builder.go:125
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.Exec(...)
          test/e2e/framework/kubectl/builder.go:107
        k8s.io/kubernetes/test/e2e/framework/kubectl.RunKubectl({0xc0030a6a00?, 0x4?}, {0xc0001478a0?, 0x29?, 0xc000147860?})
          test/e2e/framework/kubectl/builder.go:154
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmd(...)
          test/e2e/framework/pod/output/output.go:82
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmdWithRetries({0xc0030a6a00, 0x10}, {0xc0030a69ec, 0x4}, {0xc00205dc40, 0x38}, 0xc00106a820?, 0x45d964b800)
          test/e2e/framework/pod/output/output.go:105
        k8s.io/kubernetes/test/e2e/framework/statefulset.ExecInStatefulPods({0x801de88?, 0xc001e1dba0?}, 0xc000147e20?, {0xc00205dc40, 0x38})
          test/e2e/framework/statefulset/rest.go:240
      > k8s.io/kubernetes/test/e2e/apps.restoreHTTPProbe({0x801de88, 0xc001e1dba0}, 0x0?)
          test/e2e/apps/statefulset.go:1728
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:642
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select, 3 minutes]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:48:16.906: INFO: rc: 1
Nov 26 12:48:16.906: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 7m26.252s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 5m20.004s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 1m41.186s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [sleep]
        time.Sleep(0x2540be400)
          /usr/local/go/src/runtime/time.go:195
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmdWithRetries({0xc0030a6a00, 0x10}, {0xc0030a69ec, 0x4}, {0xc00205dc40, 0x38}, 0xc00106a820?, 0x45d964b800)
          test/e2e/framework/pod/output/output.go:113
        k8s.io/kubernetes/test/e2e/framework/statefulset.ExecInStatefulPods({0x801de88?, 0xc001e1dba0?}, 0xc000147e20?, {0xc00205dc40, 0x38})
          test/e2e/framework/statefulset/rest.go:240
      > k8s.io/kubernetes/test/e2e/apps.restoreHTTPProbe({0x801de88, 0xc001e1dba0}, 0x0?)
          test/e2e/apps/statefulset.go:1728
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:642
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select, 3 minutes]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:48:26.906: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:48:27.592: INFO: rc: 1
Nov 26 12:48:27.592: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 7m46.257s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 5m40.008s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 2m1.19s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [sleep]
        time.Sleep(0x2540be400)
          /usr/local/go/src/runtime/time.go:195
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmdWithRetries({0xc0030a6a00, 0x10}, {0xc0030a69ec, 0x4}, {0xc00205dc40, 0x38}, 0xc00106a820?, 0x45d964b800)
          test/e2e/framework/pod/output/output.go:113
        k8s.io/kubernetes/test/e2e/framework/statefulset.ExecInStatefulPods({0x801de88?, 0xc001e1dba0?}, 0xc000147e20?, {0xc00205dc40, 0x38})
          test/e2e/framework/statefulset/rest.go:240
      > k8s.io/kubernetes/test/e2e/apps.restoreHTTPProbe({0x801de88, 0xc001e1dba0}, 0x0?)
          test/e2e/apps/statefulset.go:1728
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:642
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select, 3 minutes]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:48:37.592: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:48:38.058: INFO: rc: 1
Nov 26 12:48:38.058: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
Nov 26 12:48:48.059: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:48:48.635: INFO: rc: 1
Nov 26 12:48:48.635: INFO: Waiting 10s to retry failed RunHostCmd: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("webserver")

error:
exit status 1
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 8m6.259s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 6m0.011s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 2m21.193s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [sleep]
        time.Sleep(0x2540be400)
          /usr/local/go/src/runtime/time.go:195
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmdWithRetries({0xc0030a6a00, 0x10}, {0xc0030a69ec, 0x4}, {0xc00205dc40, 0x38}, 0xc00106a820?, 0x45d964b800)
          test/e2e/framework/pod/output/output.go:113
        k8s.io/kubernetes/test/e2e/framework/statefulset.ExecInStatefulPods({0x801de88?, 0xc001e1dba0?}, 0xc000147e20?, {0xc00205dc40, 0x38})
          test/e2e/framework/statefulset/rest.go:240
      > k8s.io/kubernetes/test/e2e/apps.restoreHTTPProbe({0x801de88, 0xc001e1dba0}, 0x0?)
          test/e2e/apps/statefulset.go:1728
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:642
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select, 4 minutes]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:48:58.636: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 exec ss-0 -- /bin/sh -x -c mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true'
Nov 26 12:48:59.442: INFO: stderr: "+ mv -v /tmp/index.html /usr/local/apache2/htdocs/\nmv: can't rename '/tmp/index.html': No such file or directory\n+ true\n"
Nov 26 12:48:59.442: INFO: stdout: "'/tmp/index.html' -> '/usr/local/apache2/htdocs/index.html'\n"
Nov 26 12:48:59.442: INFO: stdout of mv -v /tmp/index.html /usr/local/apache2/htdocs/ || true on ss-0: '/tmp/index.html' -> '/usr/local/apache2/htdocs/index.html'

Nov 26 12:48:59.521: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:49:09.574: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 8m26.261s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 6m20.013s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 2m41.195s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:49:19.577: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:49:29.572: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 8m46.264s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 6m40.015s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 3m1.198s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:49:39.576: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:49:49.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 9m6.267s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 7m0.019s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 3m21.201s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:49:59.565: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:50:09.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 9m26.27s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 7m20.021s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 3m41.203s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:50:19.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:50:29.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 9m46.272s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 7m40.023s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 4m1.205s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:50:39.584: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:50:49.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 10m6.274s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 8m0.026s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 4m21.208s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select, 2 minutes]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:50:59.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:51:09.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 10m26.278s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 8m20.029s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 4m41.211s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select, 2 minutes]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:51:19.565: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:51:29.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 10m46.281s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 8m40.032s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 5m1.215s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select, 2 minutes]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:51:39.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:51:49.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 11m6.284s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 9m0.036s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 5m21.218s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:51:59.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:52:09.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 11m26.291s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 9m20.042s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 5m41.224s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:52:19.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:52:29.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 11m46.303s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 9m40.054s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 6m1.236s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Goroutines of Interest
      goroutine 12303 [select]
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.UntilWithoutRetry({0x7fe0c00, 0xc0031bf620}, {0x7fbcaa0, 0xc006198d00}, {0xc0000cdf38, 0x1, 0x2?})
          vendor/k8s.io/client-go/tools/watch/until.go:73
        k8s.io/kubernetes/vendor/k8s.io/client-go/tools/watch.Until({0x7fe0c00, 0xc0031bf620}, {0xc0030a6bd8?, 0x75b5158?}, {0x7facee0?, 0xc0017caa50?}, {0xc0000cdf38, 0x1, 0x1})
          vendor/k8s.io/client-go/tools/watch/until.go:114
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10.2()
          test/e2e/apps/statefulset.go:613
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10
          test/e2e/apps/statefulset.go:605
------------------------------
Nov 26 12:52:39.565: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:52:49.566: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 12m6.305s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 10m0.057s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 6m21.239s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:52:59.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:53:09.565: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 12m26.308s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 10m20.059s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 6m41.241s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:53:19.565: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:53:29.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 12m46.311s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 10m40.062s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 7m1.244s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:53:39.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:53:49.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 13m6.312s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 11m0.064s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 7m21.246s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:53:59.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:54:09.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 13m26.314s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 11m20.066s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 7m41.248s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:54:19.565: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:54:29.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 13m46.316s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 11m40.068s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 8m1.25s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:54:39.566: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:54:49.565: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 14m6.318s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 12m0.07s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 8m21.252s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:54:59.564: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:55:09.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 14m26.32s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 12m20.072s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 8m41.254s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:55:19.581: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:55:29.616: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 14m46.322s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 12m40.074s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 9m1.256s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:55:39.578: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:55:49.599: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 15m6.324s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 13m0.076s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 9m21.258s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:55:59.589: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:56:09.597: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 15m26.327s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 13m20.078s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 9m41.26s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:56:19.605: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:56:29.579: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 15m46.329s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 13m40.081s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 10m1.263s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:56:39.587: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:56:49.595: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 16m6.331s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 14m0.083s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 10m21.265s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:56:59.565: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:57:09.564: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 16m26.333s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 14m20.084s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 10m41.266s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:57:19.568: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:57:29.565: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 16m46.335s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 14m40.087s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 11m1.269s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:57:39.579: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:57:49.586: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 17m6.337s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 15m0.089s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 11m21.271s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:57:59.579: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:58:09.622: INFO: Found 2 stateful pods, waiting for 3
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 17m26.339s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 15m20.091s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 11m41.273s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:58:19.582: INFO: Found 2 stateful pods, waiting for 3
Nov 26 12:58:29.577: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:58:29.577: INFO: Waiting for pod ss-1 to enter Running - Ready=true, currently Running - Ready=true
Nov 26 12:58:29.577: INFO: Waiting for pod ss-2 to enter Running - Ready=true, currently Pending - Ready=false
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 17m46.341s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 15m40.092s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 12m1.274s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:58:39.570: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=false
Nov 26 12:58:49.595: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=false
------------------------------
Progress Report for Ginkgo Process #15
Automatically polling progress:
  [sig-apps] StatefulSet Basic StatefulSet functionality [StatefulSetBasic] Scaling should happen in predictable order and halt if any stateful pod is unhealthy [Slow] [Conformance] (Spec Runtime: 18m6.343s)
    test/e2e/apps/statefulset.go:587
    In [It] (Node Runtime: 16m0.095s)
      test/e2e/apps/statefulset.go:587
      At [By Step] Scaling up stateful set ss to 3 replicas and waiting until all of them will be running in namespace statefulset-5414 (Step Runtime: 12m21.277s)
        test/e2e/apps/statefulset.go:641

      Spec Goroutine
      goroutine 12301 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0045f22e8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x90?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x1?, 0xc000147de0?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x801de88?, 0xc001e1dba0?, 0xc000147e20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
          test/e2e/framework/statefulset/wait.go:35
        k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
          test/e2e/framework/statefulset/wait.go:80
      > k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
          test/e2e/apps/statefulset.go:643
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc002f92600})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:58:59.603: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=false
Nov 26 12:58:59.666: INFO: Waiting for pod ss-0 to enter Running - Ready=true, currently Running - Ready=false
Nov 26 12:58:59.666: FAIL: Failed waiting for pods to enter running: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunning({0x801de88?, 0xc001e1dba0}, 0x3, 0x3, 0xc0009af900)
	test/e2e/framework/statefulset/wait.go:58 +0xf9
k8s.io/kubernetes/test/e2e/framework/statefulset.WaitForRunningAndReady(...)
	test/e2e/framework/statefulset/wait.go:80
k8s.io/kubernetes/test/e2e/apps.glob..func10.2.10()
	test/e2e/apps/statefulset.go:643 +0x6d0
[AfterEach] Basic StatefulSet functionality [StatefulSetBasic]
  test/e2e/apps/statefulset.go:124
Nov 26 12:58:59.717: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 describe po ss-0'
Nov 26 12:59:00.110: INFO: stderr: ""
Nov 26 12:59:00.110: INFO: stdout: "Name:             ss-0\nNamespace:        statefulset-5414\nPriority:         0\nService Account:  default\nNode:             bootstrap-e2e-minion-group-tvmd/10.138.0.3\nStart Time:       Sat, 26 Nov 2022 12:42:57 +0000\nLabels:           baz=blah\n                  controller-revision-hash=ss-7b6c9599d5\n                  foo=bar\n                  statefulset.kubernetes.io/pod-name=ss-0\nAnnotations:      <none>\nStatus:           Running\nIP:               10.64.2.49\nIPs:\n  IP:           10.64.2.49\nControlled By:  StatefulSet/ss\nContainers:\n  webserver:\n    Container ID:   containerd://54af61a3fdfd2999880a9aa2d0aa7e411f3108a863e9775030e925b6b3271bc6\n    Image:          registry.k8s.io/e2e-test-images/httpd:2.4.38-4\n    Image ID:       registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22\n    Port:           <none>\n    Host Port:      <none>\n    State:          Waiting\n      Reason:       CrashLoopBackOff\n    Last State:     Terminated\n      Reason:       Completed\n      Exit Code:    0\n      Started:      Sat, 26 Nov 2022 12:58:28 +0000\n      Finished:     Sat, 26 Nov 2022 12:58:28 +0000\n    Ready:          False\n    Restart Count:  7\n    Readiness:      http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1\n    Environment:    <none>\n    Mounts:\n      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-v4n88 (ro)\nConditions:\n  Type              Status\n  Initialized       True \n  Ready             False \n  ContainersReady   False \n  PodScheduled      True \nVolumes:\n  kube-api-access-v4n88:\n    Type:                    Projected (a volume that contains injected data from multiple sources)\n    TokenExpirationSeconds:  3607\n    ConfigMapName:           kube-root-ca.crt\n    ConfigMapOptional:       <nil>\n    DownwardAPI:             true\nQoS Class:                   BestEffort\nNode-Selectors:              <none>\nTolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s\n                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s\nEvents:\n  Type     Reason          Age                  From               Message\n  ----     ------          ----                 ----               -------\n  Normal   Scheduled       16m                  default-scheduler  Successfully assigned statefulset-5414/ss-0 to bootstrap-e2e-minion-group-tvmd\n  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get \"http://10.64.2.138:80/index.html\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get \"http://10.64.2.140:80/index.html\": dial tcp 10.64.2.140:80: i/o timeout (Client.Timeout exceeded while awaiting headers)\n  Normal   Pulled          15m (x3 over 16m)    kubelet            Container image \"registry.k8s.io/e2e-test-images/httpd:2.4.38-4\" already present on machine\n  Normal   Created         15m (x3 over 16m)    kubelet            Created container webserver\n  Normal   Started         15m (x3 over 16m)    kubelet            Started container webserver\n  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get \"http://10.64.2.142:80/index.html\": read tcp 10.64.2.1:60346->10.64.2.142:80: read: connection reset by peer\n  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get \"http://10.64.2.142:80/index.html\": dial tcp 10.64.2.142:80: connect: connection refused\n  Normal   SandboxChanged  15m (x3 over 15m)    kubelet            Pod sandbox changed, it will be killed and re-created.\n  Normal   Killing         5m39s (x7 over 15m)  kubelet            Stopping container webserver\n  Warning  BackOff         60s (x59 over 15m)   kubelet            Back-off restarting failed container webserver in pod ss-0_statefulset-5414(4325014a-5f3e-4fc7-839d-7c573747950a)\n"
Nov 26 12:59:00.110: INFO: 
Output of kubectl describe ss-0:
Name:             ss-0
Namespace:        statefulset-5414
Priority:         0
Service Account:  default
Node:             bootstrap-e2e-minion-group-tvmd/10.138.0.3
Start Time:       Sat, 26 Nov 2022 12:42:57 +0000
Labels:           baz=blah
                  controller-revision-hash=ss-7b6c9599d5
                  foo=bar
                  statefulset.kubernetes.io/pod-name=ss-0
Annotations:      <none>
Status:           Running
IP:               10.64.2.49
IPs:
  IP:           10.64.2.49
Controlled By:  StatefulSet/ss
Containers:
  webserver:
    Container ID:   containerd://54af61a3fdfd2999880a9aa2d0aa7e411f3108a863e9775030e925b6b3271bc6
    Image:          registry.k8s.io/e2e-test-images/httpd:2.4.38-4
    Image ID:       registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sat, 26 Nov 2022 12:58:28 +0000
      Finished:     Sat, 26 Nov 2022 12:58:28 +0000
    Ready:          False
    Restart Count:  7
    Readiness:      http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-v4n88 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  kube-api-access-v4n88:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason          Age                  From               Message
  ----     ------          ----                 ----               -------
  Normal   Scheduled       16m                  default-scheduler  Successfully assigned statefulset-5414/ss-0 to bootstrap-e2e-minion-group-tvmd
  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get "http://10.64.2.138:80/index.html": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get "http://10.64.2.140:80/index.html": dial tcp 10.64.2.140:80: i/o timeout (Client.Timeout exceeded while awaiting headers)
  Normal   Pulled          15m (x3 over 16m)    kubelet            Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
  Normal   Created         15m (x3 over 16m)    kubelet            Created container webserver
  Normal   Started         15m (x3 over 16m)    kubelet            Started container webserver
  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get "http://10.64.2.142:80/index.html": read tcp 10.64.2.1:60346->10.64.2.142:80: read: connection reset by peer
  Warning  Unhealthy       15m                  kubelet            Readiness probe failed: Get "http://10.64.2.142:80/index.html": dial tcp 10.64.2.142:80: connect: connection refused
  Normal   SandboxChanged  15m (x3 over 15m)    kubelet            Pod sandbox changed, it will be killed and re-created.
  Normal   Killing         5m39s (x7 over 15m)  kubelet            Stopping container webserver
  Warning  BackOff         60s (x59 over 15m)   kubelet            Back-off restarting failed container webserver in pod ss-0_statefulset-5414(4325014a-5f3e-4fc7-839d-7c573747950a)

Nov 26 12:59:00.110: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 logs ss-0 --tail=100'
Nov 26 12:59:00.624: INFO: rc: 1
Nov 26 12:59:00.624: INFO: 
Last 100 log lines of ss-0:

Nov 26 12:59:00.625: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 describe po ss-1'
Nov 26 12:59:01.130: INFO: stderr: ""
Nov 26 12:59:01.130: INFO: stdout: "Name:             ss-1\nNamespace:        statefulset-5414\nPriority:         0\nService Account:  default\nNode:             bootstrap-e2e-minion-group-dldd/10.138.0.4\nStart Time:       Sat, 26 Nov 2022 12:51:43 +0000\nLabels:           baz=blah\n                  controller-revision-hash=ss-7b6c9599d5\n                  foo=bar\n                  statefulset.kubernetes.io/pod-name=ss-1\nAnnotations:      <none>\nStatus:           Running\nIP:               10.64.1.223\nIPs:\n  IP:           10.64.1.223\nControlled By:  StatefulSet/ss\nContainers:\n  webserver:\n    Container ID:   containerd://383e87b733738ae122f05e46c729d522c9896a5199132922174405d0124ca4bf\n    Image:          registry.k8s.io/e2e-test-images/httpd:2.4.38-4\n    Image ID:       registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22\n    Port:           <none>\n    Host Port:      <none>\n    State:          Running\n      Started:      Sat, 26 Nov 2022 12:57:19 +0000\n    Last State:     Terminated\n      Reason:       Completed\n      Exit Code:    0\n      Started:      Sat, 26 Nov 2022 12:54:05 +0000\n      Finished:     Sat, 26 Nov 2022 12:56:35 +0000\n    Ready:          True\n    Restart Count:  4\n    Readiness:      http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1\n    Environment:    <none>\n    Mounts:\n      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6kfw (ro)\nConditions:\n  Type              Status\n  Initialized       True \n  Ready             True \n  ContainersReady   True \n  PodScheduled      True \nVolumes:\n  kube-api-access-h6kfw:\n    Type:                    Projected (a volume that contains injected data from multiple sources)\n    TokenExpirationSeconds:  3607\n    ConfigMapName:           kube-root-ca.crt\n    ConfigMapOptional:       <nil>\n    DownwardAPI:             true\nQoS Class:                   BestEffort\nNode-Selectors:              <none>\nTolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s\n                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s\nEvents:\n  Type     Reason          Age                    From               Message\n  ----     ------          ----                   ----               -------\n  Normal   Scheduled       7m17s                  default-scheduler  Successfully assigned statefulset-5414/ss-1 to bootstrap-e2e-minion-group-dldd\n  Warning  Unhealthy       7m13s                  kubelet            Readiness probe failed: Get \"http://10.64.1.187:80/index.html\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n  Warning  Unhealthy       7m10s                  kubelet            Readiness probe failed: Get \"http://10.64.1.188:80/index.html\": dial tcp 10.64.1.188:80: connect: connection refused\n  Normal   Pulled          6m55s (x3 over 7m17s)  kubelet            Container image \"registry.k8s.io/e2e-test-images/httpd:2.4.38-4\" already present on machine\n  Normal   Created         6m55s (x3 over 7m17s)  kubelet            Created container webserver\n  Normal   Started         6m55s (x3 over 7m17s)  kubelet            Started container webserver\n  Normal   Killing         5m25s (x3 over 7m14s)  kubelet            Stopping container webserver\n  Normal   SandboxChanged  5m25s (x3 over 7m13s)  kubelet            Pod sandbox changed, it will be killed and re-created.\n  Warning  Unhealthy       5m24s                  kubelet            Readiness probe failed: Get \"http://10.64.1.189:80/index.html\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n  Warning  BackOff         2m9s (x11 over 7m9s)   kubelet            Back-off restarting failed container webserver in pod ss-1_statefulset-5414(ffe1bc64-6568-484c-a6f5-53f3fd2a5ead)\n"
Nov 26 12:59:01.131: INFO: 
Output of kubectl describe ss-1:
Name:             ss-1
Namespace:        statefulset-5414
Priority:         0
Service Account:  default
Node:             bootstrap-e2e-minion-group-dldd/10.138.0.4
Start Time:       Sat, 26 Nov 2022 12:51:43 +0000
Labels:           baz=blah
                  controller-revision-hash=ss-7b6c9599d5
                  foo=bar
                  statefulset.kubernetes.io/pod-name=ss-1
Annotations:      <none>
Status:           Running
IP:               10.64.1.223
IPs:
  IP:           10.64.1.223
Controlled By:  StatefulSet/ss
Containers:
  webserver:
    Container ID:   containerd://383e87b733738ae122f05e46c729d522c9896a5199132922174405d0124ca4bf
    Image:          registry.k8s.io/e2e-test-images/httpd:2.4.38-4
    Image ID:       registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sat, 26 Nov 2022 12:57:19 +0000
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sat, 26 Nov 2022 12:54:05 +0000
      Finished:     Sat, 26 Nov 2022 12:56:35 +0000
    Ready:          True
    Restart Count:  4
    Readiness:      http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6kfw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-h6kfw:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason          Age                    From               Message
  ----     ------          ----                   ----               -------
  Normal   Scheduled       7m17s                  default-scheduler  Successfully assigned statefulset-5414/ss-1 to bootstrap-e2e-minion-group-dldd
  Warning  Unhealthy       7m13s                  kubelet            Readiness probe failed: Get "http://10.64.1.187:80/index.html": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy       7m10s                  kubelet            Readiness probe failed: Get "http://10.64.1.188:80/index.html": dial tcp 10.64.1.188:80: connect: connection refused
  Normal   Pulled          6m55s (x3 over 7m17s)  kubelet            Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
  Normal   Created         6m55s (x3 over 7m17s)  kubelet            Created container webserver
  Normal   Started         6m55s (x3 over 7m17s)  kubelet            Started container webserver
  Normal   Killing         5m25s (x3 over 7m14s)  kubelet            Stopping container webserver
  Normal   SandboxChanged  5m25s (x3 over 7m13s)  kubelet            Pod sandbox changed, it will be killed and re-created.
  Warning  Unhealthy       5m24s                  kubelet            Readiness probe failed: Get "http://10.64.1.189:80/index.html": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Warning  BackOff         2m9s (x11 over 7m9s)   kubelet            Back-off restarting failed container webserver in pod ss-1_statefulset-5414(ffe1bc64-6568-484c-a6f5-53f3fd2a5ead)

Nov 26 12:59:01.131: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 logs ss-1 --tail=100'
Nov 26 12:59:01.539: INFO: rc: 1
Nov 26 12:59:01.539: INFO: 
Last 100 log lines of ss-1:

Nov 26 12:59:01.539: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 describe po ss-2'
Nov 26 12:59:01.972: INFO: stderr: ""
Nov 26 12:59:01.972: INFO: stdout: "Name:             ss-2\nNamespace:        statefulset-5414\nPriority:         0\nService Account:  default\nNode:             <none>\nLabels:           baz=blah\n                  controller-revision-hash=ss-7b6c9599d5\n                  foo=bar\n                  statefulset.kubernetes.io/pod-name=ss-2\nAnnotations:      <none>\nStatus:           Pending\nIP:               \nIPs:              <none>\nControlled By:    StatefulSet/ss\nContainers:\n  webserver:\n    Image:        registry.k8s.io/e2e-test-images/httpd:2.4.38-4\n    Port:         <none>\n    Host Port:    <none>\n    Readiness:    http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1\n    Environment:  <none>\n    Mounts:\n      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cd5s6 (ro)\nVolumes:\n  kube-api-access-cd5s6:\n    Type:                    Projected (a volume that contains injected data from multiple sources)\n    TokenExpirationSeconds:  3607\n    ConfigMapName:           kube-root-ca.crt\n    ConfigMapOptional:       <nil>\n    DownwardAPI:             true\nQoS Class:                   BestEffort\nNode-Selectors:              <none>\nTolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s\n                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s\nEvents:                      <none>\n"
Nov 26 12:59:01.972: INFO: 
Output of kubectl describe ss-2:
Name:             ss-2
Namespace:        statefulset-5414
Priority:         0
Service Account:  default
Node:             <none>
Labels:           baz=blah
                  controller-revision-hash=ss-7b6c9599d5
                  foo=bar
                  statefulset.kubernetes.io/pod-name=ss-2
Annotations:      <none>
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    StatefulSet/ss
Containers:
  webserver:
    Image:        registry.k8s.io/e2e-test-images/httpd:2.4.38-4
    Port:         <none>
    Host Port:    <none>
    Readiness:    http-get http://:80/index.html delay=0s timeout=1s period=1s #success=1 #failure=1
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cd5s6 (ro)
Volumes:
  kube-api-access-cd5s6:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

Nov 26 12:59:01.972: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=statefulset-5414 logs ss-2 --tail=100'
Nov 26 12:59:02.290: INFO: stderr: ""
Nov 26 12:59:02.290: INFO: stdout: ""
Nov 26 12:59:02.290: INFO: 
Last 100 log lines of ss-2:

Nov 26 12:59:02.290: INFO: Deleting all statefulset in ns statefulset-5414
Nov 26 12:59:02.336: INFO: Scaling statefulset ss to 0
Nov 26 13:03:42.677: INFO: Waiting for statefulset status.replicas updated to 0
Nov 26 13:03:42.745: INFO: Deleting statefulset ss
[AfterEach] [sig-apps] StatefulSet
  test/e2e/framework/node/init/init.go:32
Nov 26 13:03:43.072: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-apps] StatefulSet
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-apps] StatefulSet
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 13:03:43.144
STEP: Collecting events from namespace "statefulset-5414". 11/26/22 13:03:43.144
STEP: Found 34 events. 11/26/22 13:03:43.219
Nov 26 13:03:43.219: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for ss-0: { } Scheduled: Successfully assigned statefulset-5414/ss-0 to bootstrap-e2e-minion-group-tvmd
Nov 26 13:03:43.219: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for ss-1: { } Scheduled: Successfully assigned statefulset-5414/ss-1 to bootstrap-e2e-minion-group-dldd
Nov 26 13:03:43.219: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for ss-2: { } Scheduled: Successfully assigned statefulset-5414/ss-2 to bootstrap-e2e-minion-group-8pmq
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:42:57 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Pod ss-0 in StatefulSet ss successful
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:42:59 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:42:59 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:42:59 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:43:01 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Killing: Stopping container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:43:02 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:43:03 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Unhealthy: Readiness probe failed: Get "http://10.64.2.138:80/index.html": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:43:07 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} BackOff: Back-off restarting failed container webserver in pod ss-0_statefulset-5414(4325014a-5f3e-4fc7-839d-7c573747950a)
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:43:07 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Unhealthy: Readiness probe failed: Get "http://10.64.2.140:80/index.html": dial tcp 10.64.2.140:80: i/o timeout (Client.Timeout exceeded while awaiting headers)
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:43:23 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Unhealthy: Readiness probe failed: Get "http://10.64.2.142:80/index.html": dial tcp 10.64.2.142:80: connect: connection refused
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:43:23 +0000 UTC - event for ss-0: {kubelet bootstrap-e2e-minion-group-tvmd} Unhealthy: Readiness probe failed: Get "http://10.64.2.142:80/index.html": read tcp 10.64.2.1:60346->10.64.2.142:80: read: connection reset by peer
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:48:57 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Pod ss-1 in StatefulSet ss successful
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:44 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:44 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} Created: Created container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:44 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} Started: Started container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:47 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} Killing: Stopping container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:48 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:48 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} Unhealthy: Readiness probe failed: Get "http://10.64.1.187:80/index.html": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:51 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} Unhealthy: Readiness probe failed: Get "http://10.64.1.188:80/index.html": dial tcp 10.64.1.188:80: connect: connection refused
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:51:52 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} BackOff: Back-off restarting failed container webserver in pod ss-1_statefulset-5414(ffe1bc64-6568-484c-a6f5-53f3fd2a5ead)
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:53:37 +0000 UTC - event for ss-1: {kubelet bootstrap-e2e-minion-group-dldd} Unhealthy: Readiness probe failed: Get "http://10.64.1.189:80/index.html": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 26 13:03:43.219: INFO: At 2022-11-26 12:58:28 +0000 UTC - event for ss: {statefulset-controller } SuccessfulCreate: create Pod ss-2 in StatefulSet ss successful
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:01:57 +0000 UTC - event for ss-2: {kubelet bootstrap-e2e-minion-group-8pmq} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-cd5s6" : failed to sync configmap cache: timed out waiting for the condition
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:01:58 +0000 UTC - event for ss-2: {kubelet bootstrap-e2e-minion-group-8pmq} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:01:58 +0000 UTC - event for ss-2: {kubelet bootstrap-e2e-minion-group-8pmq} Started: Started container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:01:58 +0000 UTC - event for ss-2: {kubelet bootstrap-e2e-minion-group-8pmq} Created: Created container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:01:59 +0000 UTC - event for ss: {statefulset-controller } SuccessfulDelete: delete Pod ss-2 in StatefulSet ss successful
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:01:59 +0000 UTC - event for ss-2: {kubelet bootstrap-e2e-minion-group-8pmq} Killing: Stopping container webserver
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:02:01 +0000 UTC - event for ss-2: {kubelet bootstrap-e2e-minion-group-8pmq} Unhealthy: Readiness probe failed: Get "http://10.64.0.191:80/index.html": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:03:30 +0000 UTC - event for ss: {statefulset-controller } SuccessfulDelete: delete Pod ss-1 in StatefulSet ss successful
Nov 26 13:03:43.219: INFO: At 2022-11-26 13:03:31 +0000 UTC - event for ss: {statefulset-controller } SuccessfulDelete: delete Pod ss-0 in StatefulSet ss successful
Nov 26 13:03:43.288: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 26 13:03:43.288: INFO: 
Nov 26 13:03:43.396: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 13:03:43.484: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 16505 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 13:02:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:21 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:21 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:21 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 13:02:21 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 13:03:43.484: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 13:03:43.575: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 13:03:43.658: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 26 13:03:43.658: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 13:03:43.730: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 16797 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1783":"bootstrap-e2e-minion-group-8pmq"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:55:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-26 13:01:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 13:03:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 13:01:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 13:01:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 13:01:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 13:01:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 13:00:47 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 13:00:47 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 13:00:47 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 13:00:47 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 13:03:43.730: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 13:03:43.810: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 13:03:43.876: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-8pmq: error trying to reach service: No agent available
Nov 26 13:03:43.876: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 13:03:43.928: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 16571 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1382":"csi-mock-csi-mock-volumes-1382"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:56:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-26 13:01:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 13:02:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 13:01:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 13:01:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 13:01:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 13:01:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 13:02:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 13:03:43.928: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 13:03:43.990: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 13:03:44.077: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-dldd: error trying to reach service: No agent available
Nov 26 13:03:44.077: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 13:03:44.133: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 16472 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-5061":"bootstrap-e2e-minion-group-tvmd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-26 13:01:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 13:02:11 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 13:02:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 13:01:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 13:01:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 13:01:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 13:01:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 13:01:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:11 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:11 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 13:02:11 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 13:02:11 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},},Config:nil,},}
Nov 26 13:03:44.133: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 13:03:44.192: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 13:03:44.318: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-tvmd: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-apps] StatefulSet
  tear down framework | framework.go:193
STEP: Destroying namespace "statefulset-5414" for this suite. 11/26/22 13:03:44.318

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-auth] ServiceAccounts should support InClusterConfig with token rotation [Slow] 1m2s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-auth\]\sServiceAccounts\sshould\ssupport\sInClusterConfig\swith\stoken\srotation\s\[Slow\]$'

test/e2e/auth/service_accounts.go:497
k8s.io/kubernetes/test/e2e/auth.glob..func5.6()
	test/e2e/auth/service_accounts.go:497 +0x877
				[BeforeEach] [sig-auth] ServiceAccounts
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:55:41.79
Nov 26 12:55:41.790: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename svcaccounts 11/26/22 12:55:41.792
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:55:42.117
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:55:42.275
[BeforeEach] [sig-auth] ServiceAccounts
  test/e2e/framework/metrics/init/init.go:31
[It] should support InClusterConfig with token rotation [Slow]
  test/e2e/auth/service_accounts.go:432
Nov 26 12:55:42.591: INFO: created pod
Nov 26 12:55:42.591: INFO: Waiting up to 1m0s for 1 pods to be running and ready: [inclusterclient]
Nov 26 12:55:42.591: INFO: Waiting up to 1m0s for pod "inclusterclient" in namespace "svcaccounts-8734" to be "running and ready"
Nov 26 12:55:42.651: INFO: Pod "inclusterclient": Phase="Pending", Reason="", readiness=false. Elapsed: 59.964956ms
Nov 26 12:55:42.651: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:55:44.915: INFO: Pod "inclusterclient": Phase="Pending", Reason="", readiness=false. Elapsed: 2.324815154s
Nov 26 12:55:44.915: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:55:46.748: INFO: Pod "inclusterclient": Phase="Pending", Reason="", readiness=false. Elapsed: 4.157467292s
Nov 26 12:55:46.748: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:55:48.729: INFO: Pod "inclusterclient": Phase="Pending", Reason="", readiness=false. Elapsed: 6.13851661s
Nov 26 12:55:48.729: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:55:50.774: INFO: Pod "inclusterclient": Phase="Pending", Reason="", readiness=false. Elapsed: 8.183040701s
Nov 26 12:55:50.774: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:55:52.763: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 10.172222805s
Nov 26 12:55:52.763: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:55:54.780: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 12.189758622s
Nov 26 12:55:54.780: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:55:56.745: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 14.154748687s
Nov 26 12:55:56.745: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:55:58.785: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 16.194256967s
Nov 26 12:55:58.785: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:00.787: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 18.196198679s
Nov 26 12:56:00.787: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:02.756: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 20.165364432s
Nov 26 12:56:02.756: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:04.730: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 22.139046605s
Nov 26 12:56:04.730: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:06.730: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 24.13931297s
Nov 26 12:56:06.730: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:08.749: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 26.158464876s
Nov 26 12:56:08.749: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:10.732: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 28.141456826s
Nov 26 12:56:10.732: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:12.741: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 30.150467396s
Nov 26 12:56:12.741: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:14.721: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 32.130711547s
Nov 26 12:56:14.721: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:16.753: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 34.162065391s
Nov 26 12:56:16.753: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:18.733: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 36.142857725s
Nov 26 12:56:18.733: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:20.806: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 38.21494995s
Nov 26 12:56:20.806: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:22.712: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 40.121871974s
Nov 26 12:56:22.712: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:24.724: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 42.133692965s
Nov 26 12:56:24.724: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:27.089: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 44.498652353s
Nov 26 12:56:27.089: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:28.725: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 46.134670041s
Nov 26 12:56:28.725: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:30.715: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 48.124452958s
Nov 26 12:56:30.715: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:32.736: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 50.145204849s
Nov 26 12:56:32.736: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:34.754: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 52.163592951s
Nov 26 12:56:34.754: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:36.751: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 54.160785803s
Nov 26 12:56:36.751: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:38.724: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 56.133174458s
Nov 26 12:56:38.724: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:40.985: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 58.394180518s
Nov 26 12:56:40.985: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:42.739: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 1m0.148592989s
Nov 26 12:56:42.739: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:42.802: INFO: Pod "inclusterclient": Phase="Failed", Reason="", readiness=false. Elapsed: 1m0.211485238s
Nov 26 12:56:42.802: INFO: Error evaluating pod condition running and ready: want pod 'inclusterclient' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Failed'
Nov 26 12:56:42.802: INFO: Pod inclusterclient failed to be running and ready.
Nov 26 12:56:42.802: INFO: Wanted all 1 pods to be running and ready. Result: false. Pods: [inclusterclient]
Nov 26 12:56:42.802: FAIL: pod "inclusterclient" in ns "svcaccounts-8734" never became ready

Full Stack Trace
k8s.io/kubernetes/test/e2e/auth.glob..func5.6()
	test/e2e/auth/service_accounts.go:497 +0x877
[AfterEach] [sig-auth] ServiceAccounts
  test/e2e/framework/node/init/init.go:32
Nov 26 12:56:42.803: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-auth] ServiceAccounts
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-auth] ServiceAccounts
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:56:42.903
STEP: Collecting events from namespace "svcaccounts-8734". 11/26/22 12:56:42.904
STEP: Found 5 events. 11/26/22 12:56:42.956
Nov 26 12:56:42.956: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for inclusterclient: { } Scheduled: Successfully assigned svcaccounts-8734/inclusterclient to bootstrap-e2e-minion-group-tvmd
Nov 26 12:56:42.956: INFO: At 2022-11-26 12:55:47 +0000 UTC - event for inclusterclient: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:56:42.956: INFO: At 2022-11-26 12:55:47 +0000 UTC - event for inclusterclient: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container inclusterclient
Nov 26 12:56:42.956: INFO: At 2022-11-26 12:55:47 +0000 UTC - event for inclusterclient: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container inclusterclient
Nov 26 12:56:42.956: INFO: At 2022-11-26 12:55:47 +0000 UTC - event for inclusterclient: {kubelet bootstrap-e2e-minion-group-tvmd} Killing: Stopping container inclusterclient
Nov 26 12:56:43.016: INFO: POD              NODE                             PHASE   GRACE  CONDITIONS
Nov 26 12:56:43.016: INFO: inclusterclient  bootstrap-e2e-minion-group-tvmd  Failed         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:55:42 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:55:48 +0000 UTC PodFailed } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:55:48 +0000 UTC PodFailed } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:55:42 +0000 UTC  }]
Nov 26 12:56:43.016: INFO: 
Nov 26 12:56:43.101: INFO: Unable to fetch svcaccounts-8734/inclusterclient/inclusterclient logs: an error on the server ("unknown") has prevented the request from succeeding (get pods inclusterclient)
Nov 26 12:56:43.170: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:56:43.240: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 11767 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:52:08 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:56:43.240: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:56:43.317: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:56:43.407: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 26 12:56:43.407: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:56:43.468: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 14114 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1783":"bootstrap-e2e-minion-group-8pmq"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:55:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:56:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-11-26 12:56:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:56:43.469: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:56:43.531: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:56:43.619: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-8pmq: error trying to reach service: No agent available
Nov 26 12:56:43.619: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:56:43.671: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 14130 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-1932":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-1645":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-7472":"bootstrap-e2e-minion-group-dldd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:56:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:56:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {node-problem-detector Update v1 2022-11-26 12:56:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:19 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:19 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:19 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:56:19 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7472^b49a987a-6d89-11ed-ac3e-26f046335973],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:56:43.671: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:56:43.737: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:56:43.837: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-dldd: error trying to reach service: No agent available
Nov 26 12:56:43.837: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:56:43.899: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 14156 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-1174":"bootstrap-e2e-minion-group-tvmd","csi-hostpath-multivolume-2222":"bootstrap-e2e-minion-group-tvmd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-26 12:56:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:56:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {node-problem-detector Update v1 2022-11-26 12:56:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:13 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:13 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:13 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:56:13 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-multivolume-2222^998cb992-6d89-11ed-95ee-766acc69df1a kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-2222^998cb992-6d89-11ed-95ee-766acc69df1a,DevicePath:,},},Config:nil,},}
Nov 26 12:56:43.900: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:56:44.039: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:56:44.273: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-tvmd: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-auth] ServiceAccounts
  tear down framework | framework.go:193
STEP: Destroying namespace "svcaccounts-8734" for this suite. 11/26/22 12:56:44.273

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should return command exit codes [Slow] running a failing command with --leave-stdin-open 37s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\sreturn\scommand\sexit\scodes\s\[Slow\]\srunning\sa\sfailing\scommand\swith\s\-\-leave\-stdin\-open$'

test/e2e/kubectl/kubectl.go:589
k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.7.7()
	test/e2e/kubectl/kubectl.go:589 +0x22d
				[BeforeEach] [sig-cli] Kubectl client
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:32:30.794
Nov 26 12:32:30.794: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename kubectl 11/26/22 12:32:30.796
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:32:30.947
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:32:31.031
[BeforeEach] [sig-cli] Kubectl client
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-cli] Kubectl client
  test/e2e/kubectl/kubectl.go:274
[BeforeEach] Simple pod
  test/e2e/kubectl/kubectl.go:411
STEP: creating the pod from  11/26/22 12:32:31.111
Nov 26 12:32:31.112: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7336 create -f -'
Nov 26 12:32:31.644: INFO: stderr: ""
Nov 26 12:32:31.644: INFO: stdout: "pod/httpd created\n"
Nov 26 12:32:31.644: INFO: Waiting up to 5m0s for 1 pods to be running and ready: [httpd]
Nov 26 12:32:31.644: INFO: Waiting up to 5m0s for pod "httpd" in namespace "kubectl-7336" to be "running and ready"
Nov 26 12:32:31.686: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 41.405157ms
Nov 26 12:32:31.686: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:32:33.730: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2.085783559s
Nov 26 12:32:33.730: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:32:35.732: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4.087722443s
Nov 26 12:32:35.732: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:32:37.734: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 6.090109787s
Nov 26 12:32:37.734: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:32:39.758: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 8.11335284s
Nov 26 12:32:39.758: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:32:41.740: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 10.096197791s
Nov 26 12:32:41.740: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:32:43.752: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 12.107953103s
Nov 26 12:32:43.752: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  }]
Nov 26 12:32:45.736: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 14.091838325s
Nov 26 12:32:45.736: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  }]
Nov 26 12:32:47.729: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 16.084409916s
Nov 26 12:32:47.729: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  }]
Nov 26 12:32:49.737: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 18.092815413s
Nov 26 12:32:49.737: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  }]
Nov 26 12:32:51.766: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 20.121748137s
Nov 26 12:32:51.766: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  }]
Nov 26 12:32:53.747: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 22.102919256s
Nov 26 12:32:53.747: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  }]
Nov 26 12:32:55.768: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 24.124252365s
Nov 26 12:32:55.768: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:31 +0000 UTC  }]
Nov 26 12:32:57.735: INFO: Pod "httpd": Phase="Running", Reason="", readiness=true. Elapsed: 26.091077705s
Nov 26 12:32:57.735: INFO: Pod "httpd" satisfied condition "running and ready"
Nov 26 12:32:57.735: INFO: Wanted all 1 pods to be running and ready. Result: true. Pods: [httpd]
[It] [Slow] running a failing command with --leave-stdin-open
  test/e2e/kubectl/kubectl.go:585
Nov 26 12:32:57.735: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7336 run -i --image=registry.k8s.io/e2e-test-images/busybox:1.29-4 --restart=Never --pod-running-timeout=2m0s failure-4 --leave-stdin-open -- /bin/sh -c exit 42'
Nov 26 12:33:05.996: INFO: rc: 1
Nov 26 12:33:05.996: INFO: Unexpected error: 
    <exec.CodeExitError>: {
        Err: <*errors.errorString | 0xc0010b06f0>{
            s: "error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7336 run -i --image=registry.k8s.io/e2e-test-images/busybox:1.29-4 --restart=Never --pod-running-timeout=2m0s failure-4 --leave-stdin-open -- /bin/sh -c exit 42:\nCommand stdout:\n\nstderr:\nError from server: Get \"https://10.138.0.3:10250/containerLogs/kubectl-7336/failure-4/failure-4\": No agent available\n\nerror:\nexit status 1",
        },
        Code: 1,
    }
Nov 26 12:33:05.996: FAIL: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7336 run -i --image=registry.k8s.io/e2e-test-images/busybox:1.29-4 --restart=Never --pod-running-timeout=2m0s failure-4 --leave-stdin-open -- /bin/sh -c exit 42:
Command stdout:

stderr:
Error from server: Get "https://10.138.0.3:10250/containerLogs/kubectl-7336/failure-4/failure-4": No agent available

error:
exit status 1

Full Stack Trace
k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.7.7()
	test/e2e/kubectl/kubectl.go:589 +0x22d
[AfterEach] Simple pod
  test/e2e/kubectl/kubectl.go:417
STEP: using delete to clean up resources 11/26/22 12:33:05.996
Nov 26 12:33:05.996: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7336 delete --grace-period=0 --force -f -'
Nov 26 12:33:06.622: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n"
Nov 26 12:33:06.622: INFO: stdout: "pod \"httpd\" force deleted\n"
Nov 26 12:33:06.622: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7336 get rc,svc -l name=httpd --no-headers'
Nov 26 12:33:06.918: INFO: stderr: "No resources found in kubectl-7336 namespace.\n"
Nov 26 12:33:06.918: INFO: stdout: ""
Nov 26 12:33:06.918: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7336 get pods -l name=httpd -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}'
Nov 26 12:33:07.156: INFO: stderr: ""
Nov 26 12:33:07.156: INFO: stdout: ""
[AfterEach] [sig-cli] Kubectl client
  test/e2e/framework/node/init/init.go:32
Nov 26 12:33:07.156: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-cli] Kubectl client
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-cli] Kubectl client
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:33:07.263
STEP: Collecting events from namespace "kubectl-7336". 11/26/22 12:33:07.263
STEP: Found 10 events. 11/26/22 12:33:07.32
Nov 26 12:33:07.321: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for failure-4: { } Scheduled: Successfully assigned kubectl-7336/failure-4 to bootstrap-e2e-minion-group-tvmd
Nov 26 12:33:07.321: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for httpd: { } Scheduled: Successfully assigned kubectl-7336/httpd to bootstrap-e2e-minion-group-tvmd
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:32:37 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Pulling: Pulling image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4"
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:32:42 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" in 5.176515751s (5.176536426s including waiting)
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:32:42 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container httpd
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:32:42 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container httpd
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:32:59 +0000 UTC - event for failure-4: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:32:59 +0000 UTC - event for failure-4: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container failure-4
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:32:59 +0000 UTC - event for failure-4: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container failure-4
Nov 26 12:33:07.321: INFO: At 2022-11-26 12:33:06 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Killing: Stopping container httpd
Nov 26 12:33:07.381: INFO: POD        NODE                             PHASE   GRACE  CONDITIONS
Nov 26 12:33:07.381: INFO: failure-4  bootstrap-e2e-minion-group-tvmd  Failed         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:57 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:57 +0000 UTC PodFailed } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:57 +0000 UTC PodFailed } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:32:57 +0000 UTC  }]
Nov 26 12:33:07.381: INFO: 
Nov 26 12:33:07.455: INFO: Unable to fetch kubectl-7336/failure-4/failure-4 logs: an error on the server ("unknown") has prevented the request from succeeding (get pods failure-4)
Nov 26 12:33:07.515: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:33:07.603: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 622 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:31:42 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:31:42 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:31:42 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:31:42 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:31:42 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:33:07.604: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:33:07.673: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:33:07.753: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 26 12:33:07.753: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:33:07.814: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 1483 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:31:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:32:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:31:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:31:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:31:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:31:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:32:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:33:07.814: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:33:07.872: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:33:07.962: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-8pmq: error trying to reach service: No agent available
Nov 26 12:33:07.962: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:33:08.036: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 1509 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-3254":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-1645":"bootstrap-e2e-minion-group-dldd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-26 12:31:23 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:32:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:32:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:31:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:31:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:31:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:31:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:50 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:50 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:50 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:32:50 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-multivolume-3254^6f21557e-6d86-11ed-a5d7-5e4cd5cae8b7 kubernetes.io/csi/csi-mock-csi-mock-volumes-1645^6c5b1ee9-6d86-11ed-a529-b2dae19d89b3],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1645^6c5b1ee9-6d86-11ed-a529-b2dae19d89b3,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-3254^6f21557e-6d86-11ed-a5d7-5e4cd5cae8b7,DevicePath:,},},Config:nil,},}
Nov 26 12:33:08.037: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:33:08.102: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:33:08.183: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-dldd: error trying to reach service: No agent available
Nov 26 12:33:08.183: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:33:08.248: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 1533 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-8901":"csi-mock-csi-mock-volumes-8901"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-26 12:31:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:32:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:31:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:31:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:31:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:31:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:31:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:32:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:32:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:33:08.249: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:33:08.326: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:33:08.409: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-tvmd: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-cli] Kubectl client
  tear down framework | framework.go:193
STEP: Destroying namespace "kubectl-7336" for this suite. 11/26/22 12:33:08.409

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-cli] Kubectl client Simple pod should return command exit codes [Slow] running a failing command without --restart=Never, but with --rm 17s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\sreturn\scommand\sexit\scodes\s\[Slow\]\srunning\sa\sfailing\scommand\swithout\s\-\-restart\=Never\,\sbut\swith\s\-\-rm$'

test/e2e/kubectl/kubectl.go:580
k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.7.6()
	test/e2e/kubectl/kubectl.go:580 +0x36a
				[BeforeEach] [sig-cli] Kubectl client
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:44:38.484
Nov 26 12:44:38.485: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename kubectl 11/26/22 12:44:38.487
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:44:38.662
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:44:38.751
[BeforeEach] [sig-cli] Kubectl client
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-cli] Kubectl client
  test/e2e/kubectl/kubectl.go:274
[BeforeEach] Simple pod
  test/e2e/kubectl/kubectl.go:411
STEP: creating the pod from  11/26/22 12:44:38.844
Nov 26 12:44:38.844: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7648 create -f -'
Nov 26 12:44:39.460: INFO: stderr: ""
Nov 26 12:44:39.460: INFO: stdout: "pod/httpd created\n"
Nov 26 12:44:39.460: INFO: Waiting up to 5m0s for 1 pods to be running and ready: [httpd]
Nov 26 12:44:39.460: INFO: Waiting up to 5m0s for pod "httpd" in namespace "kubectl-7648" to be "running and ready"
Nov 26 12:44:39.533: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 73.37913ms
Nov 26 12:44:39.533: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:44:41.586: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 2.125880694s
Nov 26 12:44:41.586: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:44:43.585: INFO: Pod "httpd": Phase="Pending", Reason="", readiness=false. Elapsed: 4.124808374s
Nov 26 12:44:43.585: INFO: Error evaluating pod condition running and ready: want pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' to be 'Running' but was 'Pending'
Nov 26 12:44:45.648: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 6.187957445s
Nov 26 12:44:45.648: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC  }]
Nov 26 12:44:47.584: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 8.124206696s
Nov 26 12:44:47.584: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC  }]
Nov 26 12:44:49.579: INFO: Pod "httpd": Phase="Running", Reason="", readiness=false. Elapsed: 10.119046617s
Nov 26 12:44:49.579: INFO: Error evaluating pod condition running and ready: pod 'httpd' on 'bootstrap-e2e-minion-group-tvmd' didn't have condition {Ready True}; conditions: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC ContainersNotReady containers with unready status: [httpd]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:39 +0000 UTC  }]
Nov 26 12:44:51.575: INFO: Pod "httpd": Phase="Running", Reason="", readiness=true. Elapsed: 12.115270793s
Nov 26 12:44:51.575: INFO: Pod "httpd" satisfied condition "running and ready"
Nov 26 12:44:51.575: INFO: Wanted all 1 pods to be running and ready. Result: true. Pods: [httpd]
[It] [Slow] running a failing command without --restart=Never, but with --rm
  test/e2e/kubectl/kubectl.go:571
Nov 26 12:44:51.575: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7648 run -i --image=registry.k8s.io/e2e-test-images/busybox:1.29-4 --restart=OnFailure --rm --pod-running-timeout=2m0s failure-3 -- /bin/sh -c cat && exit 42'
Nov 26 12:44:53.974: INFO: rc: 1
Nov 26 12:44:53.974: FAIL: Missing expected 'timed out' error, got: exec.CodeExitError{Err:(*errors.errorString)(0xc0010151d0), Code:1}

Full Stack Trace
k8s.io/kubernetes/test/e2e/kubectl.glob..func1.8.7.6()
	test/e2e/kubectl/kubectl.go:580 +0x36a
[AfterEach] Simple pod
  test/e2e/kubectl/kubectl.go:417
STEP: using delete to clean up resources 11/26/22 12:44:53.974
Nov 26 12:44:53.975: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7648 delete --grace-period=0 --force -f -'
Nov 26 12:44:54.218: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n"
Nov 26 12:44:54.218: INFO: stdout: "pod \"httpd\" force deleted\n"
Nov 26 12:44:54.219: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7648 get rc,svc -l name=httpd --no-headers'
Nov 26 12:44:54.459: INFO: stderr: "No resources found in kubectl-7648 namespace.\n"
Nov 26 12:44:54.459: INFO: stdout: ""
Nov 26 12:44:54.459: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=kubectl-7648 get pods -l name=httpd -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}'
Nov 26 12:44:54.662: INFO: stderr: ""
Nov 26 12:44:54.662: INFO: stdout: ""
[AfterEach] [sig-cli] Kubectl client
  test/e2e/framework/node/init/init.go:32
Nov 26 12:44:54.662: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-cli] Kubectl client
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-cli] Kubectl client
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:44:54.713
STEP: Collecting events from namespace "kubectl-7648". 11/26/22 12:44:54.713
STEP: Found 10 events. 11/26/22 12:44:54.768
Nov 26 12:44:54.768: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for failure-3: { } Scheduled: Successfully assigned kubectl-7648/failure-3 to bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:54.768: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for httpd: { } Scheduled: Successfully assigned kubectl-7648/httpd to bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:40 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:40 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container httpd
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:40 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container httpd
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:43 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} Killing: Stopping container httpd
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:44 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-tvmd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:53 +0000 UTC - event for failure-3: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:53 +0000 UTC - event for failure-3: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container failure-3
Nov 26 12:44:54.768: INFO: At 2022-11-26 12:44:53 +0000 UTC - event for failure-3: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container failure-3
Nov 26 12:44:54.814: INFO: POD        NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:44:54.814: INFO: failure-3  bootstrap-e2e-minion-group-tvmd  Running  30s    [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:51 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:53 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:53 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:44:51 +0000 UTC  }]
Nov 26 12:44:54.814: INFO: 
Nov 26 12:44:54.877: INFO: Unable to fetch kubectl-7648/failure-3/failure-3 logs: an error on the server ("unknown") has prevented the request from succeeding (get pods failure-3)
Nov 26 12:44:54.925: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:44:54.967: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 6333 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:41:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:44:54.967: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:44:55.015: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:44:55.064: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 26 12:44:55.064: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:44:55.113: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 9477 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-2975":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-multivolume-5002":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-multivolume-7037":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-1783":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-3960":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-6237":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-7979":"bootstrap-e2e-minion-group-8pmq"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-26 12:41:23 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:44:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:44:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:44:46 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:44:46 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:44:46 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:44:46 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-multivolume-5002^f6ec6f89-6d87-11ed-8904-52ce12031d16 kubernetes.io/csi/csi-hostpath-multivolume-5002^f7c6a89c-6d87-11ed-8904-52ce12031d16],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f7c6a89c-6d87-11ed-8904-52ce12031d16,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f6ec6f89-6d87-11ed-8904-52ce12031d16,DevicePath:,},},Config:nil,},}
Nov 26 12:44:55.113: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:44:55.185: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:44:55.270: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-8pmq: error trying to reach service: No agent available
Nov 26 12:44:55.270: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:44:55.339: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 9166 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-7800":"bootstrap-e2e-minion-group-dldd","csi-hostpath-volumeio-6147":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-1645":"bootstrap-e2e-minion-group-dldd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-26 12:41:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:43:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:44:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:44:55.340: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:44:55.437: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:44:55.518: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-dldd: error trying to reach service: No agent available
Nov 26 12:44:55.518: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:55.581: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 9422 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-3438":"bootstrap-e2e-minion-group-tvmd","csi-hostpath-provisioning-7215":"bootstrap-e2e-minion-group-tvmd","csi-mock-csi-mock-volumes-9921":"bootstrap-e2e-minion-group-tvmd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-26 12:41:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:43:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:44:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead kubernetes.io/csi/csi-mock-csi-mock-volumes-9921^dee9b0e7-6d87-11ed-8ad2-265b8ea29070],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},},Config:nil,},}
Nov 26 12:44:55.581: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:55.660: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:55.750: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-tvmd: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-cli] Kubectl client
  tear down framework | framework.go:193
STEP: Destroying namespace "kubectl-7648" for this suite. 11/26/22 12:44:55.75

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-network] LoadBalancers ESIPP [Slow] should handle updates to ExternalTrafficPolicy field 3m25s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sLoadBalancers\sESIPP\s\[Slow\]\sshould\shandle\supdates\sto\sExternalTrafficPolicy\sfield$'

test/e2e/network/loadbalancer.go:1492
k8s.io/kubernetes/test/e2e/network.glob..func20.7()
	test/e2e/network/loadbalancer.go:1492 +0x155
				[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:56:07.808
Nov 26 12:56:07.808: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename esipp 11/26/22 12:56:07.81
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:56:07.97
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:56:08.083
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1250
[It] should handle updates to ExternalTrafficPolicy field
  test/e2e/network/loadbalancer.go:1480
STEP: creating a service esipp-7614/external-local-update with type=LoadBalancer 11/26/22 12:56:08.482
STEP: setting ExternalTrafficPolicy=Local 11/26/22 12:56:08.482
STEP: waiting for loadbalancer for service esipp-7614/external-local-update 11/26/22 12:56:08.671
Nov 26 12:56:08.671: INFO: Waiting up to 15m0s for service "external-local-update" to have a LoadBalancer
STEP: creating a pod to be part of the service external-local-update 11/26/22 12:57:30.808
Nov 26 12:57:30.874: INFO: Waiting up to 2m0s for 1 pods to be created
Nov 26 12:57:30.923: INFO: Found all 1 pods
Nov 26 12:57:30.923: INFO: Waiting up to 2m0s for 1 pods to be running and ready: [external-local-update-5h72n]
Nov 26 12:57:30.923: INFO: Waiting up to 2m0s for pod "external-local-update-5h72n" in namespace "esipp-7614" to be "running and ready"
Nov 26 12:57:30.967: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 44.568525ms
Nov 26 12:57:30.967: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:33.008: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 2.085688675s
Nov 26 12:57:33.008: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:35.009: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 4.086584476s
Nov 26 12:57:35.009: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:37.009: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 6.086245579s
Nov 26 12:57:37.009: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:39.033: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110429284s
Nov 26 12:57:39.033: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:41.081: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 10.158411208s
Nov 26 12:57:41.081: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:43.044: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 12.121863121s
Nov 26 12:57:43.045: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:45.021: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 14.098692455s
Nov 26 12:57:45.021: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:47.034: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 16.111007455s
Nov 26 12:57:47.034: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:49.021: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 18.098273108s
Nov 26 12:57:49.021: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:51.032: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 20.109107034s
Nov 26 12:57:51.032: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:53.020: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 22.097283681s
Nov 26 12:57:53.020: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:55.029: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 24.106310055s
Nov 26 12:57:55.029: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:57.039: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 26.116516964s
Nov 26 12:57:57.039: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:57:59.041: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 28.118622491s
Nov 26 12:57:59.041: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:01.047: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 30.124727473s
Nov 26 12:58:01.047: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:03.016: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 32.092952769s
Nov 26 12:58:03.016: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:05.031: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 34.108709222s
Nov 26 12:58:05.031: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:07.057: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 36.133918286s
Nov 26 12:58:07.057: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:09.035: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 38.112553154s
Nov 26 12:58:09.035: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:11.015: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 40.091932361s
Nov 26 12:58:11.015: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:13.030: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 42.107139761s
Nov 26 12:58:13.030: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:15.018: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 44.095346491s
Nov 26 12:58:15.018: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:17.026: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 46.103407859s
Nov 26 12:58:17.026: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:19.019: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 48.096239701s
Nov 26 12:58:19.019: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:21.085: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 50.162269624s
Nov 26 12:58:21.085: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:23.063: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 52.140334758s
Nov 26 12:58:23.063: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:25.137: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 54.214075042s
Nov 26 12:58:25.137: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:27.033: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 56.110280918s
Nov 26 12:58:27.033: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:29.017: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 58.094390616s
Nov 26 12:58:29.017: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:31.031: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.108486658s
Nov 26 12:58:31.031: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:33.018: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.095278546s
Nov 26 12:58:33.018: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:35.099: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.17653334s
Nov 26 12:58:35.099: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:37.033: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.110663942s
Nov 26 12:58:37.033: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:39.025: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.101889914s
Nov 26 12:58:39.025: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:41.029: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.106524233s
Nov 26 12:58:41.029: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:44.183: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m13.260038606s
Nov 26 12:58:44.183: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:45.040: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.117685591s
Nov 26 12:58:45.040: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:47.012: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.089474584s
Nov 26 12:58:47.012: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:49.088: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.165572213s
Nov 26 12:58:49.088: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:51.022: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.099292252s
Nov 26 12:58:51.022: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:53.024: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.101496826s
Nov 26 12:58:53.024: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:55.047: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.124014037s
Nov 26 12:58:55.047: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:57.025: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.102204819s
Nov 26 12:58:57.025: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:58:59.022: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.099600164s
Nov 26 12:58:59.022: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:01.041: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.118227515s
Nov 26 12:59:01.041: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:03.029: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.106651788s
Nov 26 12:59:03.029: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:05.039: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.116341637s
Nov 26 12:59:05.039: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:07.062: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.139434163s
Nov 26 12:59:07.062: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:09.053: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.130588072s
Nov 26 12:59:09.053: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:11.013: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.090599959s
Nov 26 12:59:11.013: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:13.018: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.094938895s
Nov 26 12:59:13.018: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:15.024: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.10107294s
Nov 26 12:59:15.024: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:17.036: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.112973051s
Nov 26 12:59:17.036: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:19.138: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.215797023s
Nov 26 12:59:19.138: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:21.106: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.18310706s
Nov 26 12:59:21.106: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:23.016: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.093187964s
Nov 26 12:59:23.016: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:25.020: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.097523417s
Nov 26 12:59:25.020: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:27.098: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.174915687s
Nov 26 12:59:27.098: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:29.018: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.095097941s
Nov 26 12:59:29.018: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:31.026: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.103588232s
Nov 26 12:59:31.026: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:31.084: INFO: Pod "external-local-update-5h72n": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.161267168s
Nov 26 12:59:31.084: INFO: Error evaluating pod condition running and ready: want pod 'external-local-update-5h72n' on '' to be 'Running' but was 'Pending'
Nov 26 12:59:31.084: INFO: Pod external-local-update-5h72n failed to be running and ready.
Nov 26 12:59:31.084: INFO: Wanted all 1 pods to be running and ready. Result: false. Pods: [external-local-update-5h72n]
Nov 26 12:59:31.084: INFO: Unexpected error: 
    <*errors.errorString | 0xc0034f4dc0>: {
        s: "failed waiting for pods to be running: timeout waiting for 1 pods to be ready",
    }
Nov 26 12:59:31.084: FAIL: failed waiting for pods to be running: timeout waiting for 1 pods to be ready

Full Stack Trace
k8s.io/kubernetes/test/e2e/network.glob..func20.7()
	test/e2e/network/loadbalancer.go:1492 +0x155
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/node/init/init.go:32
Nov 26 12:59:31.084: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1260
Nov 26 12:59:31.208: INFO: 
Output of kubectl describe svc:

Nov 26 12:59:31.208: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-7614 describe svc --namespace=esipp-7614'
Nov 26 12:59:31.686: INFO: stderr: ""
Nov 26 12:59:31.686: INFO: stdout: "Name:                     external-local-update\nNamespace:                esipp-7614\nLabels:                   testid=external-local-update-ce646d0d-7f83-48e5-a9a5-3f2f88f9cbd6\nAnnotations:              <none>\nSelector:                 testid=external-local-update-ce646d0d-7f83-48e5-a9a5-3f2f88f9cbd6\nType:                     LoadBalancer\nIP Family Policy:         SingleStack\nIP Families:              IPv4\nIP:                       10.0.236.194\nIPs:                      10.0.236.194\nLoadBalancer Ingress:     34.168.101.193\nPort:                     <unset>  80/TCP\nTargetPort:               80/TCP\nNodePort:                 <unset>  30076/TCP\nEndpoints:                <none>\nSession Affinity:         None\nExternal Traffic Policy:  Local\nHealthCheck NodePort:     31851\nEvents:\n  Type    Reason               Age   From                Message\n  ----    ------               ----  ----                -------\n  Normal  EnsuredLoadBalancer  2m1s  service-controller  Ensured load balancer\n"
Nov 26 12:59:31.686: INFO: Name:                     external-local-update
Namespace:                esipp-7614
Labels:                   testid=external-local-update-ce646d0d-7f83-48e5-a9a5-3f2f88f9cbd6
Annotations:              <none>
Selector:                 testid=external-local-update-ce646d0d-7f83-48e5-a9a5-3f2f88f9cbd6
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.236.194
IPs:                      10.0.236.194
LoadBalancer Ingress:     34.168.101.193
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  30076/TCP
Endpoints:                <none>
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     31851
Events:
  Type    Reason               Age   From                Message
  ----    ------               ----  ----                -------
  Normal  EnsuredLoadBalancer  2m1s  service-controller  Ensured load balancer

[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:59:31.686
STEP: Collecting events from namespace "esipp-7614". 11/26/22 12:59:31.687
STEP: Found 2 events. 11/26/22 12:59:31.764
Nov 26 12:59:31.764: INFO: At 2022-11-26 12:57:30 +0000 UTC - event for external-local-update: {service-controller } EnsuredLoadBalancer: Ensured load balancer
Nov 26 12:59:31.764: INFO: At 2022-11-26 12:57:30 +0000 UTC - event for external-local-update: {replication-controller } SuccessfulCreate: Created pod: external-local-update-5h72n
Nov 26 12:59:31.923: INFO: POD                          NODE  PHASE    GRACE  CONDITIONS
Nov 26 12:59:31.923: INFO: external-local-update-5h72n        Pending         []
Nov 26 12:59:31.923: INFO: 
Nov 26 12:59:32.100: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:59:32.163: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 14816 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:57:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:57:14 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:57:14 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:57:14 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:57:14 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:59:32.163: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:59:32.231: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:59:32.307: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 26 12:59:32.307: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:59:32.367: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 15350 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1783":"bootstrap-e2e-minion-group-8pmq"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:55:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-26 12:56:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:58:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:56:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:55:40 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:59:32.368: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:59:32.439: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:59:32.520: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-8pmq: error trying to reach service: No agent available
Nov 26 12:59:32.520: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:59:32.582: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 15431 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-1932":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-1382":"csi-mock-csi-mock-volumes-1382","csi-mock-csi-mock-volumes-7472":"bootstrap-e2e-minion-group-dldd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:56:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-26 12:56:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:59:08 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:56:26 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:58:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:58:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:58:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:58:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:59:32.583: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:59:32.655: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:59:32.796: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-dldd: error trying to reach service: No agent available
Nov 26 12:59:32.796: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:59:32.894: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 15479 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-1174":"bootstrap-e2e-minion-group-tvmd","csi-hostpath-multivolume-2222":"bootstrap-e2e-minion-group-tvmd","csi-hostpath-multivolume-5061":"bootstrap-e2e-minion-group-tvmd","csi-hostpath-multivolume-8714":"bootstrap-e2e-minion-group-tvmd","csi-mock-csi-mock-volumes-8962":"csi-mock-csi-mock-volumes-8962"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-26 12:56:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:56:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:59:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:56:28 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:44 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:44 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:56:44 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:56:44 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},},Config:nil,},}
Nov 26 12:59:32.895: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:59:32.983: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:59:33.103: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-tvmd: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  tear down framework | framework.go:193
STEP: Destroying namespace "esipp-7614" for this suite. 11/26/22 12:59:33.103

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints 9m5s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sLoadBalancers\sESIPP\s\[Slow\]\sshould\sonly\starget\snodes\swith\sendpoints$'

test/e2e/framework/network/utils.go:834
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
	test/e2e/framework/network/utils.go:834 +0x545
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
	test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func20.5()
	test/e2e/network/loadbalancer.go:1382 +0x445
				[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:43:23.937
Nov 26 12:43:23.937: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename esipp 11/26/22 12:43:23.938
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:43:24.116
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:43:24.21
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1250
[It] should only target nodes with endpoints
  test/e2e/network/loadbalancer.go:1346
STEP: creating a service esipp-1647/external-local-nodes with type=LoadBalancer 11/26/22 12:43:24.506
STEP: setting ExternalTrafficPolicy=Local 11/26/22 12:43:24.506
STEP: waiting for loadbalancer for service esipp-1647/external-local-nodes 11/26/22 12:43:24.629
Nov 26 12:43:24.629: INFO: Waiting up to 15m0s for service "external-local-nodes" to have a LoadBalancer
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 5m0.506s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 5m0.001s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] waiting for loadbalancer for service esipp-1647/external-local-nodes (Step Runtime: 4m59.814s)
        test/e2e/framework/service/jig.go:260

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0017ee1c8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x10?, 0x2fd9d05?, 0x20?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0028e4de0?, 0xc000915a60?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc004fc0970?, 0x7fa7740?, 0xc00017e680?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
      > k8s.io/kubernetes/test/e2e/framework/service.(*TestJig).waitForCondition(0xc003a5e0f0, 0x4?, {0x7600fe2, 0x14}, 0x7895b68)
          test/e2e/framework/service/jig.go:631
      > k8s.io/kubernetes/test/e2e/framework/service.(*TestJig).WaitForLoadBalancer(0xc003a5e0f0, 0x44?)
          test/e2e/framework/service/jig.go:582
      > k8s.io/kubernetes/test/e2e/framework/service.(*TestJig).CreateLoadBalancerService(0xc003a5e0f0, 0x6aba880?, 0xc000915d10)
          test/e2e/framework/service/jig.go:261
      > k8s.io/kubernetes/test/e2e/framework/service.(*TestJig).CreateOnlyLocalLoadBalancerService(0xc003a5e0f0, 0x0?, 0x0, 0x0?)
          test/e2e/framework/service/jig.go:222
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1353
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
STEP: waiting for loadbalancer for service esipp-1647/external-local-nodes 11/26/22 12:48:26.938
Nov 26 12:48:26.938: INFO: Waiting up to 15m0s for service "external-local-nodes" to have a LoadBalancer
STEP: Performing setup for networking test in namespace esipp-1647 11/26/22 12:48:26.982
STEP: creating a selector 11/26/22 12:48:26.983
STEP: Creating the service pods in kubernetes 11/26/22 12:48:26.983
Nov 26 12:48:26.983: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
Nov 26 12:48:27.564: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "esipp-1647" to be "running and ready"
Nov 26 12:48:27.683: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 118.724408ms
Nov 26 12:48:27.683: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:48:29.755: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 2.190775363s
Nov 26 12:48:29.755: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:31.741: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 4.176575325s
Nov 26 12:48:31.741: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:33.737: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 6.172484984s
Nov 26 12:48:33.737: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:35.750: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 8.185827286s
Nov 26 12:48:35.750: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:37.736: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 10.17196132s
Nov 26 12:48:37.736: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:39.747: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 12.182873397s
Nov 26 12:48:39.747: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:41.763: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 14.198914977s
Nov 26 12:48:41.763: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:43.736: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 16.171487152s
Nov 26 12:48:43.736: INFO: The phase of Pod netserver-0 is Running (Ready = false)
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 5m20.508s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 5m20.003s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating the service pods in kubernetes (Step Runtime: 17.462s)
        test/e2e/framework/network/utils.go:761

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0015eaab0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x88?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf15d8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0xc005380ed0, 0xb}, {0x75ee704, 0x11}, 0x7f8f401?, 0x7895ad0)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodReadyInNamespace({0x801de88?, 0xc004572d00?}, {0xc005380ed0?, 0xc000b1aa40?}, {0xc00119faa0?, 0xc000915820?}, 0x271e5fe?)
          test/e2e/framework/pod/wait.go:564
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000b06000, {0x75c6f7c, 0x9}, 0xc001dfd6b0)
          test/e2e/framework/network/utils.go:866
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:763
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:48:45.776: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 18.212069226s
Nov 26 12:48:45.776: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:48:47.733: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=true. Elapsed: 20.168515929s
Nov 26 12:48:47.733: INFO: The phase of Pod netserver-0 is Running (Ready = true)
Nov 26 12:48:47.733: INFO: Pod "netserver-0" satisfied condition "running and ready"
Nov 26 12:48:47.779: INFO: Waiting up to 5m0s for pod "netserver-1" in namespace "esipp-1647" to be "running and ready"
Nov 26 12:48:47.834: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 54.800701ms
Nov 26 12:48:47.834: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:48:49.900: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 2.121259109s
Nov 26 12:48:49.900: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:48:51.920: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 4.140352559s
Nov 26 12:48:51.920: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:48:53.895: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 6.116237054s
Nov 26 12:48:53.895: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:48:56.006: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 8.22703288s
Nov 26 12:48:56.006: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:48:57.911: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 10.131616252s
Nov 26 12:48:57.911: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:48:59.915: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 12.135440147s
Nov 26 12:48:59.915: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:01.920: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 14.140534076s
Nov 26 12:49:01.920: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:03.887: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 16.107922269s
Nov 26 12:49:03.887: INFO: The phase of Pod netserver-1 is Running (Ready = false)
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 5m40.51s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 5m40.005s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating the service pods in kubernetes (Step Runtime: 37.464s)
        test/e2e/framework/network/utils.go:761

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc000b51bc0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x88?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf15d8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0xc003a66fc3, 0xb}, {0x75ee704, 0x11}, 0xc004fc14f0?, 0x7895ad0)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodReadyInNamespace({0x801de88?, 0xc004572d00?}, {0xc003a66fc3?, 0x0?}, {0xc00119faa0?, 0x0?}, 0xc00048d280?)
          test/e2e/framework/pod/wait.go:564
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000b06000, {0x75c6f7c, 0x9}, 0xc001dfd6b0)
          test/e2e/framework/network/utils.go:866
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:763
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:49:06.035: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 18.255596239s
Nov 26 12:49:06.035: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:07.892: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 20.112893476s
Nov 26 12:49:07.892: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:09.940: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 22.1607181s
Nov 26 12:49:09.940: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:11.970: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 24.190903746s
Nov 26 12:49:11.970: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:13.885: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 26.105430865s
Nov 26 12:49:13.885: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:15.892: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 28.113167179s
Nov 26 12:49:15.892: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:17.883: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 30.103880038s
Nov 26 12:49:17.883: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:19.884: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 32.105043152s
Nov 26 12:49:19.884: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:21.902: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 34.122487843s
Nov 26 12:49:21.902: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:23.885: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 36.106004012s
Nov 26 12:49:23.885: INFO: The phase of Pod netserver-1 is Running (Ready = false)
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 6m0.512s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 6m0.007s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating the service pods in kubernetes (Step Runtime: 57.466s)
        test/e2e/framework/network/utils.go:761

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc000b51bc0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x88?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf15d8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0xc003a66fc3, 0xb}, {0x75ee704, 0x11}, 0xc004fc14f0?, 0x7895ad0)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodReadyInNamespace({0x801de88?, 0xc004572d00?}, {0xc003a66fc3?, 0x0?}, {0xc00119faa0?, 0x0?}, 0xc00048d280?)
          test/e2e/framework/pod/wait.go:564
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000b06000, {0x75c6f7c, 0x9}, 0xc001dfd6b0)
          test/e2e/framework/network/utils.go:866
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:763
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:49:25.910: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 38.131100494s
Nov 26 12:49:25.910: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:27.889: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 40.109880669s
Nov 26 12:49:27.889: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:29.939: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 42.159822271s
Nov 26 12:49:29.939: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:31.905: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 44.125577307s
Nov 26 12:49:31.905: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:33.890: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 46.11131737s
Nov 26 12:49:33.891: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:35.887: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 48.10751756s
Nov 26 12:49:35.887: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:37.889: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 50.109528156s
Nov 26 12:49:37.889: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:39.891: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 52.111969783s
Nov 26 12:49:39.891: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:41.878: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 54.098846688s
Nov 26 12:49:41.878: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:43.880: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 56.10093324s
Nov 26 12:49:43.880: INFO: The phase of Pod netserver-1 is Running (Ready = false)
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 6m20.515s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 6m20.01s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating the service pods in kubernetes (Step Runtime: 1m17.469s)
        test/e2e/framework/network/utils.go:761

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc000b51bc0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x88?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf15d8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0xc003a66fc3, 0xb}, {0x75ee704, 0x11}, 0xc004fc14f0?, 0x7895ad0)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodReadyInNamespace({0x801de88?, 0xc004572d00?}, {0xc003a66fc3?, 0x0?}, {0xc00119faa0?, 0x0?}, 0xc00048d280?)
          test/e2e/framework/pod/wait.go:564
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000b06000, {0x75c6f7c, 0x9}, 0xc001dfd6b0)
          test/e2e/framework/network/utils.go:866
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:763
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:49:45.879: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 58.099986153s
Nov 26 12:49:45.879: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:47.876: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 1m0.096457216s
Nov 26 12:49:47.876: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:49:49.883: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=true. Elapsed: 1m2.103752993s
Nov 26 12:49:49.883: INFO: The phase of Pod netserver-1 is Running (Ready = true)
Nov 26 12:49:49.883: INFO: Pod "netserver-1" satisfied condition "running and ready"
Nov 26 12:49:49.926: INFO: Waiting up to 5m0s for pod "netserver-2" in namespace "esipp-1647" to be "running and ready"
Nov 26 12:49:49.968: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 42.313115ms
Nov 26 12:49:49.968: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:49:52.009: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 2.083224516s
Nov 26 12:49:52.009: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:49:54.009: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 4.083599837s
Nov 26 12:49:54.009: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:49:56.010: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 6.083751418s
Nov 26 12:49:56.010: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:49:58.022: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 8.095857624s
Nov 26 12:49:58.022: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:00.012: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 10.086439485s
Nov 26 12:50:00.012: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:02.010: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 12.083835773s
Nov 26 12:50:02.010: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:04.021: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 14.09535163s
Nov 26 12:50:04.021: INFO: The phase of Pod netserver-2 is Running (Ready = false)
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 6m40.517s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 6m40.013s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating the service pods in kubernetes (Step Runtime: 1m37.471s)
        test/e2e/framework/network/utils.go:761

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0019e2438, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x88?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf15d8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0xc003a67243, 0xb}, {0x75ee704, 0x11}, 0xc0016be3e0?, 0x7895ad0)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodReadyInNamespace({0x801de88?, 0xc004572d00?}, {0xc003a67243?, 0x0?}, {0xc00119faa0?, 0x0?}, 0xc00048d280?)
          test/e2e/framework/pod/wait.go:564
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000b06000, {0x75c6f7c, 0x9}, 0xc001dfd6b0)
          test/e2e/framework/network/utils.go:866
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:763
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:50:06.010: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 16.083723164s
Nov 26 12:50:06.010: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:08.010: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 18.083735791s
Nov 26 12:50:08.010: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:10.012: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 20.086208816s
Nov 26 12:50:10.012: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:12.009: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 22.083494603s
Nov 26 12:50:12.009: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:14.010: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 24.083993145s
Nov 26 12:50:14.010: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:16.010: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 26.083733221s
Nov 26 12:50:16.010: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:18.009: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 28.083630231s
Nov 26 12:50:18.009: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:20.012: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 30.086374694s
Nov 26 12:50:20.012: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:22.011: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 32.085155821s
Nov 26 12:50:22.011: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:24.019: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 34.093645758s
Nov 26 12:50:24.019: INFO: The phase of Pod netserver-2 is Running (Ready = false)
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 7m0.52s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 7m0.015s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating the service pods in kubernetes (Step Runtime: 1m57.474s)
        test/e2e/framework/network/utils.go:761

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0019e2438, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x88?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf15d8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0xc003a67243, 0xb}, {0x75ee704, 0x11}, 0xc0016be3e0?, 0x7895ad0)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodReadyInNamespace({0x801de88?, 0xc004572d00?}, {0xc003a67243?, 0x0?}, {0xc00119faa0?, 0x0?}, 0xc00048d280?)
          test/e2e/framework/pod/wait.go:564
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createNetProxyPods(0xc000b06000, {0x75c6f7c, 0x9}, 0xc001dfd6b0)
          test/e2e/framework/network/utils.go:866
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:763
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:50:26.015: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 36.088840706s
Nov 26 12:50:26.015: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:28.011: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 38.08527181s
Nov 26 12:50:28.011: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:50:30.077: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=true. Elapsed: 40.151126779s
Nov 26 12:50:30.077: INFO: The phase of Pod netserver-2 is Running (Ready = true)
Nov 26 12:50:30.077: INFO: Pod "netserver-2" satisfied condition "running and ready"
STEP: Creating test pods 11/26/22 12:50:30.148
Nov 26 12:50:30.213: INFO: Waiting up to 5m0s for pod "test-container-pod" in namespace "esipp-1647" to be "running"
Nov 26 12:50:30.264: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 51.389568ms
Nov 26 12:50:32.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 2.09329388s
Nov 26 12:50:34.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 4.092398332s
Nov 26 12:50:36.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 6.093034398s
Nov 26 12:50:38.307: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 8.093954091s
Nov 26 12:50:40.307: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 10.093965859s
Nov 26 12:50:42.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 12.092883524s
Nov 26 12:50:44.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 14.09230971s
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 7m20.522s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 7m20.017s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating test pods (Step Runtime: 14.311s)
        test/e2e/framework/network/utils.go:765

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0025ceee8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf18e8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0x75f4fa6, 0x12}, {0x75c00ca, 0x7}, 0x0?, 0x7895ad8)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodRunningInNamespace({0x801de88?, 0xc004572d00?}, {0x75f4fa6?, 0x0?}, {0xc00119faa0?, 0x0?}, 0x0?)
          test/e2e/framework/pod/wait.go:522
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodNameRunningInNamespace(...)
          test/e2e/framework/pod/wait.go:510
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createTestPods(0xc000b06000)
          test/e2e/framework/network/utils.go:727
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:766
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:50:46.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 16.092353792s
Nov 26 12:50:48.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 18.092383538s
Nov 26 12:50:50.308: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 20.095220714s
Nov 26 12:50:52.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 22.093234007s
Nov 26 12:50:54.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 24.093494378s
Nov 26 12:50:56.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 26.09292381s
Nov 26 12:50:58.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 28.09296567s
Nov 26 12:51:00.432: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 30.218861407s
Nov 26 12:51:02.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 32.092990491s
Nov 26 12:51:04.317: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 34.103688341s
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 7m40.525s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 7m40.02s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating test pods (Step Runtime: 34.314s)
        test/e2e/framework/network/utils.go:765

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0025ceee8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf18e8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0x75f4fa6, 0x12}, {0x75c00ca, 0x7}, 0x0?, 0x7895ad8)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodRunningInNamespace({0x801de88?, 0xc004572d00?}, {0x75f4fa6?, 0x0?}, {0xc00119faa0?, 0x0?}, 0x0?)
          test/e2e/framework/pod/wait.go:522
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodNameRunningInNamespace(...)
          test/e2e/framework/pod/wait.go:510
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createTestPods(0xc000b06000)
          test/e2e/framework/network/utils.go:727
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:766
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:51:06.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 36.092649908s
Nov 26 12:51:08.308: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 38.095363988s
Nov 26 12:51:10.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 40.092604661s
Nov 26 12:51:12.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 42.092928213s
Nov 26 12:51:14.307: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 44.094530103s
Nov 26 12:51:16.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 46.093006723s
Nov 26 12:51:18.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 48.092578565s
Nov 26 12:51:20.307: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 50.0942509s
Nov 26 12:51:22.307: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 52.093719762s
Nov 26 12:51:24.354: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 54.140824344s
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 8m0.527s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 8m0.022s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating test pods (Step Runtime: 54.316s)
        test/e2e/framework/network/utils.go:765

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0025ceee8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf18e8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0x75f4fa6, 0x12}, {0x75c00ca, 0x7}, 0x0?, 0x7895ad8)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodRunningInNamespace({0x801de88?, 0xc004572d00?}, {0x75f4fa6?, 0x0?}, {0xc00119faa0?, 0x0?}, 0x0?)
          test/e2e/framework/pod/wait.go:522
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodNameRunningInNamespace(...)
          test/e2e/framework/pod/wait.go:510
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createTestPods(0xc000b06000)
          test/e2e/framework/network/utils.go:727
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:766
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:51:26.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 56.092794172s
Nov 26 12:51:28.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 58.093195683s
Nov 26 12:51:30.308: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.095016336s
Nov 26 12:51:32.307: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.093941267s
Nov 26 12:51:34.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.09278542s
Nov 26 12:51:36.308: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.0954599s
Nov 26 12:51:38.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.093069726s
Nov 26 12:51:40.306: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.093433241s
Nov 26 12:51:42.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.092527514s
Nov 26 12:51:44.305: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.092307983s
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 8m20.529s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 8m20.024s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Creating test pods (Step Runtime: 1m14.318s)
        test/e2e/framework/network/utils.go:765

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0025ceee8, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x98?, 0x2fd9d05?, 0x70?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc0000820c8}, 0x75b521a?, 0xc001bf18e8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x75b6f82?, 0x4?, 0x76f3c92?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodCondition({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0x75f4fa6, 0x12}, {0x75c00ca, 0x7}, 0x0?, 0x7895ad8)
          test/e2e/framework/pod/wait.go:290
        k8s.io/kubernetes/test/e2e/framework/pod.WaitTimeoutForPodRunningInNamespace({0x801de88?, 0xc004572d00?}, {0x75f4fa6?, 0x0?}, {0xc00119faa0?, 0x0?}, 0x0?)
          test/e2e/framework/pod/wait.go:522
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodNameRunningInNamespace(...)
          test/e2e/framework/pod/wait.go:510
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).createTestPods(0xc000b06000)
          test/e2e/framework/network/utils.go:727
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setupCore(0xc000b06000, 0x7f847c442330?)
          test/e2e/framework/network/utils.go:766
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:778
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:51:46.306: INFO: Pod "test-container-pod": Phase="Running", Reason="", readiness=true. Elapsed: 1m16.092719998s
Nov 26 12:51:46.306: INFO: Pod "test-container-pod" satisfied condition "running"
Nov 26 12:51:46.346: INFO: Setting MaxTries for pod polling to 39 for networking test based on endpoint count 3
STEP: Getting node addresses 11/26/22 12:51:46.346
Nov 26 12:51:46.346: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
STEP: Creating the service on top of the pods in kubernetes 11/26/22 12:51:46.437
Nov 26 12:51:46.530: INFO: Service node-port-service in namespace esipp-1647 found.
Nov 26 12:51:46.667: INFO: Service session-affinity-service in namespace esipp-1647 found.
STEP: Waiting for NodePort service to expose endpoint 11/26/22 12:51:46.719
Nov 26 12:51:47.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:48.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:49.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:50.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:51.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:52.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:53.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:54.722: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:55.719: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:56.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:57.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:58.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:51:59.719: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:00.719: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:01.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:02.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:03.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 8m40.532s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 8m40.027s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Waiting for NodePort service to expose endpoint (Step Runtime: 17.75s)
        test/e2e/framework/network/utils.go:832

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0015ea5d0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0x20?, 0x2fd9d05?, 0x38?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollWithContext({0x7fe0bc8, 0xc0000820c8}, 0x754e980?, 0xc001bf1b70?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:460
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.Poll(0x0?, 0x0?, 0x0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:445
        k8s.io/kubernetes/test/e2e/framework.WaitForServiceEndpointsNum({0x801de88?, 0xc004572d00}, {0xc00119faa0, 0xa}, {0x75ee1b4, 0x11}, 0x3, 0x0?, 0x7f84a750e5b8?)
          test/e2e/framework/util.go:424
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:833
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:52:04.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:05.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:06.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:07.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:08.719: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:09.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:10.719: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:11.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:12.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:13.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:14.719: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:15.720: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:16.719: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:16.785: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:52:16.826: INFO: Unexpected error: failed to validate endpoints for service node-port-service in namespace: esipp-1647: 
    <*errors.errorString | 0xc00017da30>: {
        s: "timed out waiting for the condition",
    }
Nov 26 12:52:16.826: FAIL: failed to validate endpoints for service node-port-service in namespace: esipp-1647: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
	test/e2e/framework/network/utils.go:834 +0x545
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
	test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func20.5()
	test/e2e/network/loadbalancer.go:1382 +0x445
Nov 26 12:52:16.917: INFO: Waiting up to 15m0s for service "external-local-nodes" to have no LoadBalancer
------------------------------
Progress Report for Ginkgo Process #9
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should only target nodes with endpoints (Spec Runtime: 9m0.534s)
    test/e2e/network/loadbalancer.go:1346
    In [It] (Node Runtime: 9m0.029s)
      test/e2e/network/loadbalancer.go:1346
      At [By Step] Waiting for NodePort service to expose endpoint (Step Runtime: 37.751s)
        test/e2e/framework/network/utils.go:832

      Spec Goroutine
      goroutine 1296 [select]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc0000820c8}, 0xc0015eb500, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:660
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc0000820c8}, 0xa8?, 0x2fd9d05?, 0x48?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollWithContext({0x7fe0bc8, 0xc0000820c8}, 0x4?, 0xc001bf14f8?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:460
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.Poll(0x7fff4895d4f8?, 0xa?, 0x0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:445
        k8s.io/kubernetes/test/e2e/framework/providers/gce.(*Provider).EnsureLoadBalancerResourcesDeleted(0xc000e61138, {0xc005380250, 0xd}, {0xc005380628, 0x4})
          test/e2e/framework/providers/gce/gce.go:195
        k8s.io/kubernetes/test/e2e/framework.EnsureLoadBalancerResourcesDeleted(...)
          test/e2e/framework/util.go:551
        k8s.io/kubernetes/test/e2e/framework/service.(*TestJig).WaitForLoadBalancerDestroy.func1()
          test/e2e/framework/service/jig.go:602
        k8s.io/kubernetes/test/e2e/framework/service.(*TestJig).WaitForLoadBalancerDestroy(0xc003a5e0f0, {0xc005380250?, 0x0?}, 0x0?, 0x0?)
          test/e2e/framework/service/jig.go:614
        k8s.io/kubernetes/test/e2e/framework/service.(*TestJig).ChangeServiceType(0x0?, {0x75c5095?, 0x0?}, 0x0?)
          test/e2e/framework/service/jig.go:186
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5.2()
          test/e2e/network/loadbalancer.go:1365
        panic({0x70eb7e0, 0xc000563500})
          /usr/local/go/src/runtime/panic.go:884
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2.Fail({0xc001ae7290, 0x8d}, {0xc001bf1ad0?, 0x75b521a?, 0xc001bf1af0?})
          vendor/github.com/onsi/ginkgo/v2/core_dsl.go:352
        k8s.io/kubernetes/test/e2e/framework.Fail({0xc001040b00, 0x78}, {0xc001bf1b68?, 0x76740e9?, 0xc001bf1b90?})
          test/e2e/framework/log.go:61
        k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x1, {0x7fa3ee0, 0xc00017da30}, {0xc002283140?, 0x75ee1b4?, 0x11?})
          test/e2e/framework/expect.go:76
        k8s.io/kubernetes/test/e2e/framework.ExpectNoError(...)
          test/e2e/framework/expect.go:43
      > k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc000b06000, 0x3c?)
          test/e2e/framework/network/utils.go:834
      > k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001152000, {0x0, 0x0, 0xc0015cb8b0?})
          test/e2e/framework/network/utils.go:131
      > k8s.io/kubernetes/test/e2e/network.glob..func20.5()
          test/e2e/network/loadbalancer.go:1382
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000ca9e00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/node/init/init.go:32
Nov 26 12:52:27.174: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1260
Nov 26 12:52:27.226: INFO: 
Output of kubectl describe svc:

Nov 26 12:52:27.226: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-1647 describe svc --namespace=esipp-1647'
Nov 26 12:52:27.837: INFO: stderr: ""
Nov 26 12:52:27.837: INFO: stdout: "Name:              external-local-nodes\nNamespace:         esipp-1647\nLabels:            testid=external-local-nodes-d3b75e77-18ae-47ac-b5f6-4395d9ae68e4\nAnnotations:       <none>\nSelector:          testid=external-local-nodes-d3b75e77-18ae-47ac-b5f6-4395d9ae68e4\nType:              ClusterIP\nIP Family Policy:  SingleStack\nIP Families:       IPv4\nIP:                10.0.130.160\nIPs:               10.0.130.160\nPort:              <unset>  8081/TCP\nTargetPort:        80/TCP\nEndpoints:         <none>\nSession Affinity:  None\nEvents:\n  Type    Reason                Age    From                Message\n  ----    ------                ----   ----                -------\n  Normal  EnsuringLoadBalancer  8m12s  service-controller  Ensuring load balancer\n  Normal  EnsuringLoadBalancer  4m3s   service-controller  Ensuring load balancer\n  Normal  EnsuredLoadBalancer   4m1s   service-controller  Ensured load balancer\n\n\nName:                     node-port-service\nNamespace:                esipp-1647\nLabels:                   <none>\nAnnotations:              <none>\nSelector:                 selector-56497556-1ee7-4ebb-9b74-f3392581f0df=true\nType:                     NodePort\nIP Family Policy:         SingleStack\nIP Families:              IPv4\nIP:                       10.0.123.5\nIPs:                      10.0.123.5\nPort:                     http  80/TCP\nTargetPort:               8083/TCP\nNodePort:                 http  32159/TCP\nEndpoints:                <none>\nPort:                     udp  90/UDP\nTargetPort:               8081/UDP\nNodePort:                 udp  31273/UDP\nEndpoints:                <none>\nSession Affinity:         None\nExternal Traffic Policy:  Cluster\nEvents:                   <none>\n\n\nName:                     session-affinity-service\nNamespace:                esipp-1647\nLabels:                   <none>\nAnnotations:              <none>\nSelector:                 selector-56497556-1ee7-4ebb-9b74-f3392581f0df=true\nType:                     NodePort\nIP Family Policy:         SingleStack\nIP Families:              IPv4\nIP:                       10.0.205.15\nIPs:                      10.0.205.15\nPort:                     http  80/TCP\nTargetPort:               8083/TCP\nNodePort:                 http  31905/TCP\nEndpoints:                <none>\nPort:                     udp  90/UDP\nTargetPort:               8081/UDP\nNodePort:                 udp  32474/UDP\nEndpoints:                <none>\nSession Affinity:         ClientIP\nExternal Traffic Policy:  Cluster\nEvents:                   <none>\n"
Nov 26 12:52:27.837: INFO: Name:              external-local-nodes
Namespace:         esipp-1647
Labels:            testid=external-local-nodes-d3b75e77-18ae-47ac-b5f6-4395d9ae68e4
Annotations:       <none>
Selector:          testid=external-local-nodes-d3b75e77-18ae-47ac-b5f6-4395d9ae68e4
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.0.130.160
IPs:               10.0.130.160
Port:              <unset>  8081/TCP
TargetPort:        80/TCP
Endpoints:         <none>
Session Affinity:  None
Events:
  Type    Reason                Age    From                Message
  ----    ------                ----   ----                -------
  Normal  EnsuringLoadBalancer  8m12s  service-controller  Ensuring load balancer
  Normal  EnsuringLoadBalancer  4m3s   service-controller  Ensuring load balancer
  Normal  EnsuredLoadBalancer   4m1s   service-controller  Ensured load balancer


Name:                     node-port-service
Namespace:                esipp-1647
Labels:                   <none>
Annotations:              <none>
Selector:                 selector-56497556-1ee7-4ebb-9b74-f3392581f0df=true
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.123.5
IPs:                      10.0.123.5
Port:                     http  80/TCP
TargetPort:               8083/TCP
NodePort:                 http  32159/TCP
Endpoints:                <none>
Port:                     udp  90/UDP
TargetPort:               8081/UDP
NodePort:                 udp  31273/UDP
Endpoints:                <none>
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>


Name:                     session-affinity-service
Namespace:                esipp-1647
Labels:                   <none>
Annotations:              <none>
Selector:                 selector-56497556-1ee7-4ebb-9b74-f3392581f0df=true
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.205.15
IPs:                      10.0.205.15
Port:                     http  80/TCP
TargetPort:               8083/TCP
NodePort:                 http  31905/TCP
Endpoints:                <none>
Port:                     udp  90/UDP
TargetPort:               8081/UDP
NodePort:                 udp  32474/UDP
Endpoints:                <none>
Session Affinity:         ClientIP
External Traffic Policy:  Cluster
Events:                   <none>

[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:52:27.838
STEP: Collecting events from namespace "esipp-1647". 11/26/22 12:52:27.838
STEP: Found 29 events. 11/26/22 12:52:27.907
Nov 26 12:52:27.907: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-0: { } Scheduled: Successfully assigned esipp-1647/netserver-0 to bootstrap-e2e-minion-group-8pmq
Nov 26 12:52:27.907: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-1: { } Scheduled: Successfully assigned esipp-1647/netserver-1 to bootstrap-e2e-minion-group-dldd
Nov 26 12:52:27.907: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-2: { } Scheduled: Successfully assigned esipp-1647/netserver-2 to bootstrap-e2e-minion-group-tvmd
Nov 26 12:52:27.907: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for test-container-pod: { } Scheduled: Successfully assigned esipp-1647/test-container-pod to bootstrap-e2e-minion-group-dldd
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:44:15 +0000 UTC - event for external-local-nodes: {service-controller } EnsuringLoadBalancer: Ensuring load balancer
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:24 +0000 UTC - event for external-local-nodes: {service-controller } EnsuringLoadBalancer: Ensuring load balancer
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:26 +0000 UTC - event for external-local-nodes: {service-controller } EnsuredLoadBalancer: Ensured load balancer
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:28 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:28 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Created: Created container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:28 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Started: Started container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:28 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-llckj" : failed to sync configmap cache: timed out waiting for the condition
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:29 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Killing: Stopping container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:29 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Killing: Stopping container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:29 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:29 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Created: Created container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:29 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Started: Started container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:30 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:30 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:30 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:30 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:30 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:33 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} BackOff: Back-off restarting failed container webserver in pod netserver-1_esipp-1647(4ee45019-d1ae-4f56-bcd3-64a5b8c84d30)
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:49 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Killing: Stopping container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:50 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:48:55 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} BackOff: Back-off restarting failed container webserver in pod netserver-2_esipp-1647(c0443ac2-2a23-4f8a-9bf4-0fc12bcc24a8)
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:49:55 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} BackOff: Back-off restarting failed container webserver in pod netserver-0_esipp-1647(39702e5d-942c-4f15-bbf3-7e2f9dd26193)
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:51:44 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-dldd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:51:44 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-dldd} Created: Created container webserver
Nov 26 12:52:27.907: INFO: At 2022-11-26 12:51:44 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-dldd} Started: Started container webserver
Nov 26 12:52:27.952: INFO: POD                 NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:52:27.952: INFO: netserver-0         bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:48:27 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:50:28 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:50:28 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:48:27 +0000 UTC  }]
Nov 26 12:52:27.952: INFO: netserver-1         bootstrap-e2e-minion-group-dldd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:48:27 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:50:49 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:50:49 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:48:27 +0000 UTC  }]
Nov 26 12:52:27.952: INFO: netserver-2         bootstrap-e2e-minion-group-tvmd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:48:27 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:52:10 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:52:10 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:48:27 +0000 UTC  }]
Nov 26 12:52:27.952: INFO: test-container-pod  bootstrap-e2e-minion-group-dldd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:51:43 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:51:45 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:51:45 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:51:43 +0000 UTC  }]
Nov 26 12:52:27.952: INFO: 
Nov 26 12:52:28.204: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:52:28.246: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 11767 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:52:08 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:52:08 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:52:28.247: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:52:28.295: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:52:28.348: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container etcd-container ready: true, restart count 2
Nov 26 12:52:28.348: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container konnectivity-server-container ready: true, restart count 1
Nov 26 12:52:28.348: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-26 12:30:55 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container kube-addon-manager ready: true, restart count 1
Nov 26 12:52:28.348: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container kube-apiserver ready: true, restart count 1
Nov 26 12:52:28.348: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container kube-controller-manager ready: false, restart count 6
Nov 26 12:52:28.348: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container kube-scheduler ready: true, restart count 6
Nov 26 12:52:28.348: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container etcd-container ready: true, restart count 2
Nov 26 12:52:28.348: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-26 12:30:55 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container l7-lb-controller ready: true, restart count 6
Nov 26 12:52:28.348: INFO: metadata-proxy-v0.1-n9n2g started at 2022-11-26 12:31:27 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:52:28.348: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:52:28.348: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:52:28.528: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 26 12:52:28.528: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:52:28.570: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 11801 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-5002":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-1783":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-6237":"bootstrap-e2e-minion-group-8pmq","csi-mock-csi-mock-volumes-3803":"bootstrap-e2e-minion-group-8pmq"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:48:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-26 12:51:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:52:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:50:34 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f7c6a89c-6d87-11ed-8904-52ce12031d16,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f6ec6f89-6d87-11ed-8904-52ce12031d16,DevicePath:,},},Config:nil,},}
Nov 26 12:52:28.571: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:52:28.614: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:52:28.709: INFO: hostexec-bootstrap-e2e-minion-group-8pmq-8qf7k started at 2022-11-26 12:51:43 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container agnhost-container ready: true, restart count 0
Nov 26 12:52:28.709: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:44:22 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 26 12:52:28.709: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 26 12:52:28.709: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 26 12:52:28.709: INFO: 	Container hostpath ready: true, restart count 2
Nov 26 12:52:28.709: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 26 12:52:28.709: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 26 12:52:28.709: INFO: hostexec-bootstrap-e2e-minion-group-8pmq-5mxhm started at 2022-11-26 12:51:43 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container agnhost-container ready: true, restart count 0
Nov 26 12:52:28.709: INFO: konnectivity-agent-zdgr9 started at 2022-11-26 12:31:30 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container konnectivity-agent ready: false, restart count 7
Nov 26 12:52:28.709: INFO: csi-mockplugin-0 started at 2022-11-26 12:43:17 +0000 UTC (0+3 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container driver-registrar ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container mock ready: true, restart count 4
Nov 26 12:52:28.709: INFO: csi-mockplugin-0 started at 2022-11-26 12:32:33 +0000 UTC (0+4 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container busybox ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 26 12:52:28.709: INFO: 	Container driver-registrar ready: false, restart count 6
Nov 26 12:52:28.709: INFO: 	Container mock ready: false, restart count 6
Nov 26 12:52:28.709: INFO: csi-mockplugin-0 started at 2022-11-26 12:32:34 +0000 UTC (0+4 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container busybox ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 26 12:52:28.709: INFO: 	Container driver-registrar ready: false, restart count 6
Nov 26 12:52:28.709: INFO: 	Container mock ready: false, restart count 6
Nov 26 12:52:28.709: INFO: coredns-6d97d5ddb-9gsq9 started at 2022-11-26 12:31:36 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container coredns ready: false, restart count 7
Nov 26 12:52:28.709: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:43:43 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container csi-attacher ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container csi-resizer ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container csi-snapshotter ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container hostpath ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container liveness-probe ready: true, restart count 4
Nov 26 12:52:28.709: INFO: 	Container node-driver-registrar ready: true, restart count 4
Nov 26 12:52:28.709: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:44:48 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container csi-attacher ready: false, restart count 4
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 26 12:52:28.709: INFO: 	Container csi-resizer ready: false, restart count 4
Nov 26 12:52:28.709: INFO: 	Container csi-snapshotter ready: false, restart count 4
Nov 26 12:52:28.709: INFO: 	Container hostpath ready: false, restart count 4
Nov 26 12:52:28.709: INFO: 	Container liveness-probe ready: false, restart count 4
Nov 26 12:52:28.709: INFO: 	Container node-driver-registrar ready: false, restart count 4
Nov 26 12:52:28.709: INFO: metadata-proxy-v0.1-ctfqs started at 2022-11-26 12:31:19 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:52:28.709: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:52:28.709: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:37:07 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container csi-attacher ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container csi-resizer ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container csi-snapshotter ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container hostpath ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container liveness-probe ready: false, restart count 7
Nov 26 12:52:28.709: INFO: 	Container node-driver-registrar ready: false, restart count 7
Nov 26 12:52:28.709: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:39:06 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container csi-attacher ready: true, restart count 1
Nov 26 12:52:28.709: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 26 12:52:28.709: INFO: 	Container csi-resizer ready: true, restart count 1
Nov 26 12:52:28.709: INFO: 	Container csi-snapshotter ready: true, restart count 1
Nov 26 12:52:28.709: INFO: 	Container hostpath ready: true, restart count 1
Nov 26 12:52:28.709: INFO: 	Container liveness-probe ready: true, restart count 1
Nov 26 12:52:28.709: INFO: 	Container node-driver-registrar ready: true, restart count 1
Nov 26 12:52:28.709: INFO: netserver-0 started at 2022-11-26 12:48:27 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container webserver ready: false, restart count 4
Nov 26 12:52:28.709: INFO: kube-proxy-bootstrap-e2e-minion-group-8pmq started at 2022-11-26 12:31:18 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container kube-proxy ready: false, restart count 6
Nov 26 12:52:28.709: INFO: ss-0 started at 2022-11-26 12:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:28.709: INFO: 	Container webserver ready: false, restart count 5
Nov 26 12:52:28.996: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:52:28.996: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:52:29.038: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 11630 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:43:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:49:00 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-11-26 12:51:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:48:59 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:48:59 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:48:59 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:48:59 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:52:29.038: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:52:29.082: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:52:29.140: INFO: csi-mockplugin-0 started at 2022-11-26 12:43:03 +0000 UTC (0+4 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container busybox ready: true, restart count 5
Nov 26 12:52:29.140: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 26 12:52:29.140: INFO: 	Container driver-registrar ready: false, restart count 5
Nov 26 12:52:29.140: INFO: 	Container mock ready: false, restart count 5
Nov 26 12:52:29.140: INFO: ss-2 started at 2022-11-26 12:34:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container webserver ready: false, restart count 6
Nov 26 12:52:29.140: INFO: coredns-6d97d5ddb-cdh6x started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container coredns ready: true, restart count 8
Nov 26 12:52:29.140: INFO: konnectivity-agent-h9xml started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container konnectivity-agent ready: false, restart count 7
Nov 26 12:52:29.140: INFO: external-local-pods-7nqc8 started at 2022-11-26 12:39:44 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container netexec ready: true, restart count 4
Nov 26 12:52:29.140: INFO: netserver-1 started at 2022-11-26 12:48:27 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container webserver ready: true, restart count 5
Nov 26 12:52:29.140: INFO: kube-dns-autoscaler-5f6455f985-xgvqj started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container autoscaler ready: false, restart count 7
Nov 26 12:52:29.140: INFO: lb-internal-wgb6q started at 2022-11-26 12:40:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container netexec ready: false, restart count 6
Nov 26 12:52:29.140: INFO: kube-proxy-bootstrap-e2e-minion-group-dldd started at 2022-11-26 12:31:18 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container kube-proxy ready: false, restart count 7
Nov 26 12:52:29.140: INFO: metadata-proxy-v0.1-2tdgn started at 2022-11-26 12:31:18 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:52:29.140: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:52:29.140: INFO: l7-default-backend-8549d69d99-sxw6c started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container default-http-backend ready: true, restart count 0
Nov 26 12:52:29.140: INFO: csi-mockplugin-attacher-0 started at 2022-11-26 12:32:33 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container csi-attacher ready: true, restart count 5
Nov 26 12:52:29.140: INFO: ss-1 started at 2022-11-26 12:51:43 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container webserver ready: true, restart count 2
Nov 26 12:52:29.140: INFO: volume-snapshot-controller-0 started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container volume-snapshot-controller ready: false, restart count 5
Nov 26 12:52:29.140: INFO: csi-mockplugin-0 started at 2022-11-26 12:32:33 +0000 UTC (0+3 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 26 12:52:29.140: INFO: 	Container driver-registrar ready: false, restart count 5
Nov 26 12:52:29.140: INFO: 	Container mock ready: false, restart count 5
Nov 26 12:52:29.140: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:37:07 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container csi-attacher ready: false, restart count 6
Nov 26 12:52:29.140: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 26 12:52:29.140: INFO: 	Container csi-resizer ready: false, restart count 6
Nov 26 12:52:29.140: INFO: 	Container csi-snapshotter ready: false, restart count 6
Nov 26 12:52:29.140: INFO: 	Container hostpath ready: false, restart count 6
Nov 26 12:52:29.140: INFO: 	Container liveness-probe ready: false, restart count 6
Nov 26 12:52:29.140: INFO: 	Container node-driver-registrar ready: false, restart count 6
Nov 26 12:52:29.140: INFO: test-container-pod started at 2022-11-26 12:51:43 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.140: INFO: 	Container webserver ready: true, restart count 0
Nov 26 12:52:29.348: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-dldd
Nov 26 12:52:29.348: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:52:29.389: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 11712 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volumemode-3323":"bootstrap-e2e-minion-group-tvmd","csi-mock-csi-mock-volumes-9921":"bootstrap-e2e-minion-group-tvmd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-26 12:48:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-26 12:51:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:51:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:51:27 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:48:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:48:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:48:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:48:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},},Config:nil,},}
Nov 26 12:52:29.392: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:52:29.436: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:52:29.499: INFO: pause-pod-deployment-7c748f845d-rtr6l started at 2022-11-26 12:39:46 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container agnhost-pause ready: true, restart count 5
Nov 26 12:52:29.499: INFO: nfs-server started at 2022-11-26 12:34:27 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container nfs-server ready: false, restart count 6
Nov 26 12:52:29.499: INFO: kube-proxy-bootstrap-e2e-minion-group-tvmd started at 2022-11-26 12:31:20 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container kube-proxy ready: false, restart count 7
Nov 26 12:52:29.499: INFO: metadata-proxy-v0.1-qcjv9 started at 2022-11-26 12:31:20 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:52:29.499: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:52:29.499: INFO: konnectivity-agent-b72jm started at 2022-11-26 12:31:39 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container konnectivity-agent ready: true, restart count 7
Nov 26 12:52:29.499: INFO: csi-mockplugin-0 started at 2022-11-26 12:42:54 +0000 UTC (0+3 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 26 12:52:29.499: INFO: 	Container driver-registrar ready: true, restart count 2
Nov 26 12:52:29.499: INFO: 	Container mock ready: true, restart count 2
Nov 26 12:52:29.499: INFO: pvc-volume-tester-r2rw2 started at 2022-11-26 12:39:08 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container volume-tester ready: false, restart count 0
Nov 26 12:52:29.499: INFO: ss-0 started at 2022-11-26 12:42:57 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container webserver ready: true, restart count 6
Nov 26 12:52:29.499: INFO: csi-mockplugin-0 started at 2022-11-26 12:37:23 +0000 UTC (0+3 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 26 12:52:29.499: INFO: 	Container driver-registrar ready: false, restart count 5
Nov 26 12:52:29.499: INFO: 	Container mock ready: false, restart count 5
Nov 26 12:52:29.499: INFO: pvc-volume-tester-cpjl6 started at 2022-11-26 12:37:31 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container volume-tester ready: false, restart count 0
Nov 26 12:52:29.499: INFO: pvc-volume-tester-p9ctz started at 2022-11-26 12:37:44 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container volume-tester ready: false, restart count 0
Nov 26 12:52:29.499: INFO: netserver-2 started at 2022-11-26 12:48:27 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container webserver ready: false, restart count 5
Nov 26 12:52:29.499: INFO: pod-configmaps-0a5089b9-2fce-46f6-91b4-a18437a2f8f6 started at 2022-11-26 12:39:35 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container agnhost-container ready: false, restart count 0
Nov 26 12:52:29.499: INFO: pvc-volume-tester-9bf8m started at 2022-11-26 12:43:06 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container volume-tester ready: false, restart count 0
Nov 26 12:52:29.499: INFO: pod-secrets-6c7fbfed-75c0-4892-a3f4-ce4f46c517b7 started at 2022-11-26 12:48:10 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 26 12:52:29.499: INFO: mutability-test-9xppf started at 2022-11-26 12:44:04 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container netexec ready: false, restart count 5
Nov 26 12:52:29.499: INFO: metrics-server-v0.5.2-867b8754b9-4kvml started at 2022-11-26 12:31:58 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container metrics-server ready: false, restart count 6
Nov 26 12:52:29.499: INFO: 	Container metrics-server-nanny ready: false, restart count 8
Nov 26 12:52:29.499: INFO: ilb-host-exec started at 2022-11-26 12:43:25 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 26 12:52:29.499: INFO: pod-secrets-42edbd47-ddd5-4bb5-9631-c2eaa23dbda2 started at 2022-11-26 12:37:19 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 26 12:52:29.499: INFO: hostexec-bootstrap-e2e-minion-group-tvmd-8npsp started at 2022-11-26 12:37:22 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container agnhost-container ready: true, restart count 1
Nov 26 12:52:29.499: INFO: csi-mockplugin-attacher-0 started at 2022-11-26 12:37:23 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container csi-attacher ready: true, restart count 5
Nov 26 12:52:29.499: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:48:46 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container csi-attacher ready: true, restart count 4
Nov 26 12:52:29.499: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 26 12:52:29.499: INFO: 	Container csi-resizer ready: true, restart count 4
Nov 26 12:52:29.499: INFO: 	Container csi-snapshotter ready: true, restart count 4
Nov 26 12:52:29.499: INFO: 	Container hostpath ready: true, restart count 4
Nov 26 12:52:29.499: INFO: 	Container liveness-probe ready: true, restart count 4
Nov 26 12:52:29.499: INFO: 	Container node-driver-registrar ready: true, restart count 4
Nov 26 12:52:29.499: INFO: back-off-cap started at 2022-11-26 12:34:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container back-off-cap ready: false, restart count 8
Nov 26 12:52:29.499: INFO: ss-1 started at 2022-11-26 12:34:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:52:29.499: INFO: 	Container webserver ready: true, restart count 7
Nov 26 12:52:29.737: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-tvmd
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  tear down framework | framework.go:193
STEP: Destroying namespace "esipp-1647" for this suite. 11/26/22 12:52:29.738

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-network] LoadBalancers ESIPP [Slow] should work for type=LoadBalancer 4m54s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sLoadBalancers\sESIPP\s\[Slow\]\sshould\swork\sfor\stype\=LoadBalancer$'

test/e2e/framework/network/utils.go:834
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc00105c380, 0x3c?)
	test/e2e/framework/network/utils.go:834 +0x545
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc0012b6000, {0x0, 0x0, 0x0?})
	test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func20.3.1()
	test/e2e/network/loadbalancer.go:1285 +0x10a
k8s.io/kubernetes/test/e2e/network.glob..func20.3()
	test/e2e/network/loadbalancer.go:1312 +0x37f
				[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:33:31.08
Nov 26 12:33:31.080: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename esipp 11/26/22 12:33:31.081
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:33:31.36
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:33:31.473
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1250
[It] should work for type=LoadBalancer
  test/e2e/network/loadbalancer.go:1266
STEP: creating a service esipp-1787/external-local-lb with type=LoadBalancer 11/26/22 12:33:31.996
STEP: setting ExternalTrafficPolicy=Local 11/26/22 12:33:31.996
STEP: waiting for loadbalancer for service esipp-1787/external-local-lb 11/26/22 12:33:32.139
Nov 26 12:33:32.139: INFO: Waiting up to 15m0s for service "external-local-lb" to have a LoadBalancer
STEP: creating a pod to be part of the service external-local-lb 11/26/22 12:35:06.251
Nov 26 12:35:06.328: INFO: Waiting up to 2m0s for 1 pods to be created
Nov 26 12:35:06.394: INFO: Found all 1 pods
Nov 26 12:35:06.394: INFO: Waiting up to 2m0s for 1 pods to be running and ready: [external-local-lb-8s8kb]
Nov 26 12:35:06.394: INFO: Waiting up to 2m0s for pod "external-local-lb-8s8kb" in namespace "esipp-1787" to be "running and ready"
Nov 26 12:35:06.447: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 53.462935ms
Nov 26 12:35:06.447: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on '' to be 'Running' but was 'Pending'
Nov 26 12:35:08.497: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.102683016s
Nov 26 12:35:08.497: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on '' to be 'Running' but was 'Pending'
Nov 26 12:35:10.505: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110696915s
Nov 26 12:35:10.505: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on '' to be 'Running' but was 'Pending'
Nov 26 12:35:12.522: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.128305258s
Nov 26 12:35:12.522: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on '' to be 'Running' but was 'Pending'
Nov 26 12:35:14.493: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.098964811s
Nov 26 12:35:14.493: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on '' to be 'Running' but was 'Pending'
Nov 26 12:35:16.511: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.117249488s
Nov 26 12:35:16.511: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on '' to be 'Running' but was 'Pending'
Nov 26 12:35:18.509: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 12.115286028s
Nov 26 12:35:18.509: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on '' to be 'Running' but was 'Pending'
Nov 26 12:35:20.512: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 14.118231447s
Nov 26 12:35:20.512: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on 'bootstrap-e2e-minion-group-8pmq' to be 'Running' but was 'Pending'
Nov 26 12:35:22.498: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 16.104236992s
Nov 26 12:35:22.498: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on 'bootstrap-e2e-minion-group-8pmq' to be 'Running' but was 'Pending'
Nov 26 12:35:24.542: INFO: Pod "external-local-lb-8s8kb": Phase="Pending", Reason="", readiness=false. Elapsed: 18.148231932s
Nov 26 12:35:24.542: INFO: Error evaluating pod condition running and ready: want pod 'external-local-lb-8s8kb' on 'bootstrap-e2e-minion-group-8pmq' to be 'Running' but was 'Pending'
Nov 26 12:35:26.539: INFO: Pod "external-local-lb-8s8kb": Phase="Running", Reason="", readiness=true. Elapsed: 20.144725919s
Nov 26 12:35:26.539: INFO: Pod "external-local-lb-8s8kb" satisfied condition "running and ready"
Nov 26 12:35:26.539: INFO: Wanted all 1 pods to be running and ready. Result: true. Pods: [external-local-lb-8s8kb]
STEP: waiting for loadbalancer for service esipp-1787/external-local-lb 11/26/22 12:35:26.539
Nov 26 12:35:26.539: INFO: Waiting up to 15m0s for service "external-local-lb" to have a LoadBalancer
STEP: reading clientIP using the TCP service's service port via its external VIP 11/26/22 12:35:26.63
Nov 26 12:35:26.630: INFO: Poking "http://35.247.92.206:80/clientip"
Nov 26 12:35:26.671: INFO: Poke("http://35.247.92.206:80/clientip"): Get "http://35.247.92.206:80/clientip": dial tcp 35.247.92.206:80: connect: connection refused
Nov 26 12:35:28.671: INFO: Poking "http://35.247.92.206:80/clientip"
Nov 26 12:35:28.712: INFO: Poke("http://35.247.92.206:80/clientip"): Get "http://35.247.92.206:80/clientip": dial tcp 35.247.92.206:80: connect: connection refused
Nov 26 12:35:30.671: INFO: Poking "http://35.247.92.206:80/clientip"
Nov 26 12:35:30.711: INFO: Poke("http://35.247.92.206:80/clientip"): Get "http://35.247.92.206:80/clientip": dial tcp 35.247.92.206:80: connect: connection refused
Nov 26 12:35:32.672: INFO: Poking "http://35.247.92.206:80/clientip"
Nov 26 12:35:42.673: INFO: Poke("http://35.247.92.206:80/clientip"): Get "http://35.247.92.206:80/clientip": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 26 12:35:44.672: INFO: Poking "http://35.247.92.206:80/clientip"
Nov 26 12:35:44.752: INFO: Poke("http://35.247.92.206:80/clientip"): success
Nov 26 12:35:44.752: INFO: ClientIP detected by target pod using VIP:SvcPort is 34.136.112.132:35990
STEP: checking if Source IP is preserved 11/26/22 12:35:44.752
Nov 26 12:36:12.230: INFO: Waiting up to 15m0s for service "external-local-lb" to have no LoadBalancer
STEP: Performing setup for networking test in namespace esipp-1787 11/26/22 12:36:23.532
STEP: creating a selector 11/26/22 12:36:23.533
STEP: Creating the service pods in kubernetes 11/26/22 12:36:23.533
Nov 26 12:36:23.533: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
Nov 26 12:36:23.751: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "esipp-1787" to be "running and ready"
Nov 26 12:36:23.792: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 41.148032ms
Nov 26 12:36:23.792: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:25.834: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.083148057s
Nov 26 12:36:25.834: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:27.834: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.08278299s
Nov 26 12:36:27.834: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:29.860: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109254152s
Nov 26 12:36:29.860: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:31.835: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 8.083563776s
Nov 26 12:36:31.835: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:33.834: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 10.082930619s
Nov 26 12:36:33.834: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:35.835: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 12.083681815s
Nov 26 12:36:35.835: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:37.834: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 14.082854978s
Nov 26 12:36:37.834: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:39.834: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 16.083324091s
Nov 26 12:36:39.834: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:41.834: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 18.083355613s
Nov 26 12:36:41.835: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:43.837: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 20.085824888s
Nov 26 12:36:43.837: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:45.835: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 22.083517675s
Nov 26 12:36:45.835: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:47.835: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 24.08350551s
Nov 26 12:36:47.835: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:49.835: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 26.083631241s
Nov 26 12:36:49.835: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:51.835: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 28.083579601s
Nov 26 12:36:51.835: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:53.834: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 30.082455458s
Nov 26 12:36:53.834: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:55.837: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 32.086069009s
Nov 26 12:36:55.837: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:57.958: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 34.207399288s
Nov 26 12:36:57.959: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:36:59.897: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 36.146217575s
Nov 26 12:36:59.897: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:37:01.845: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 38.094260008s
Nov 26 12:37:01.845: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:37:03.924: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 40.173075689s
Nov 26 12:37:03.924: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:37:05.881: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 42.130366101s
Nov 26 12:37:05.881: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:37:07.878: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 44.126677827s
Nov 26 12:37:07.878: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:37:09.889: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 46.138325102s
Nov 26 12:37:09.889: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:11.948: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 48.197129153s
Nov 26 12:37:11.948: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:13.844: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 50.093271978s
Nov 26 12:37:13.844: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:15.919: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 52.167485861s
Nov 26 12:37:15.919: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:17.858: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 54.10711329s
Nov 26 12:37:17.858: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:19.887: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 56.136372359s
Nov 26 12:37:19.887: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:21.894: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 58.142961608s
Nov 26 12:37:21.894: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:23.856: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 1m0.104844023s
Nov 26 12:37:23.856: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:25.938: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 1m2.187108283s
Nov 26 12:37:25.938: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:37:27.835: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=true. Elapsed: 1m4.084024206s
Nov 26 12:37:27.835: INFO: The phase of Pod netserver-0 is Running (Ready = true)
Nov 26 12:37:27.835: INFO: Pod "netserver-0" satisfied condition "running and ready"
Nov 26 12:37:27.895: INFO: Waiting up to 5m0s for pod "netserver-1" in namespace "esipp-1787" to be "running and ready"
Nov 26 12:37:27.948: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 53.247053ms
Nov 26 12:37:27.948: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:30.007: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 2.112341322s
Nov 26 12:37:30.007: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:32.018: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 4.123238953s
Nov 26 12:37:32.018: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:34.019: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 6.123656256s
Nov 26 12:37:34.019: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:36.006: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 8.111101126s
Nov 26 12:37:36.006: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:38.010: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 10.114799604s
Nov 26 12:37:38.010: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:39.998: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 12.103034525s
Nov 26 12:37:39.998: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:42.025: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 14.130569968s
Nov 26 12:37:42.026: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:43.992: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 16.097046712s
Nov 26 12:37:43.992: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:45.989: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 18.094231759s
Nov 26 12:37:45.989: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:47.990: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=false. Elapsed: 20.094963375s
Nov 26 12:37:47.990: INFO: The phase of Pod netserver-1 is Running (Ready = false)
Nov 26 12:37:49.991: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=true. Elapsed: 22.096287598s
Nov 26 12:37:49.991: INFO: The phase of Pod netserver-1 is Running (Ready = true)
Nov 26 12:37:49.991: INFO: Pod "netserver-1" satisfied condition "running and ready"
Nov 26 12:37:50.034: INFO: Waiting up to 5m0s for pod "netserver-2" in namespace "esipp-1787" to be "running and ready"
Nov 26 12:37:50.110: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=true. Elapsed: 76.048427ms
Nov 26 12:37:50.110: INFO: The phase of Pod netserver-2 is Running (Ready = true)
Nov 26 12:37:50.110: INFO: Pod "netserver-2" satisfied condition "running and ready"
STEP: Creating test pods 11/26/22 12:37:50.152
Nov 26 12:37:50.207: INFO: Waiting up to 5m0s for pod "test-container-pod" in namespace "esipp-1787" to be "running"
Nov 26 12:37:50.249: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 42.240281ms
Nov 26 12:37:52.290: INFO: Pod "test-container-pod": Phase="Running", Reason="", readiness=true. Elapsed: 2.083554485s
Nov 26 12:37:52.290: INFO: Pod "test-container-pod" satisfied condition "running"
Nov 26 12:37:52.331: INFO: Setting MaxTries for pod polling to 39 for networking test based on endpoint count 3
STEP: Getting node addresses 11/26/22 12:37:52.331
Nov 26 12:37:52.331: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
STEP: Creating the service on top of the pods in kubernetes 11/26/22 12:37:52.416
Nov 26 12:37:52.511: INFO: Service node-port-service in namespace esipp-1787 found.
Nov 26 12:37:52.648: INFO: Service session-affinity-service in namespace esipp-1787 found.
STEP: Waiting for NodePort service to expose endpoint 11/26/22 12:37:52.689
Nov 26 12:37:53.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:37:54.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:37:55.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:37:56.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:37:57.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:37:58.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:37:59.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:00.689: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:01.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:02.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:03.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:04.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:05.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:06.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:07.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:08.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:09.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:10.691: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:11.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:12.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:13.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:14.691: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:15.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:16.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:17.691: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:18.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:19.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:20.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:21.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:22.690: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:22.732: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:38:22.773: INFO: Unexpected error: failed to validate endpoints for service node-port-service in namespace: esipp-1787: 
    <*errors.errorString | 0xc000187d50>: {
        s: "timed out waiting for the condition",
    }
Nov 26 12:38:22.773: FAIL: failed to validate endpoints for service node-port-service in namespace: esipp-1787: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc00105c380, 0x3c?)
	test/e2e/framework/network/utils.go:834 +0x545
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc0012b6000, {0x0, 0x0, 0x0?})
	test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func20.3.1()
	test/e2e/network/loadbalancer.go:1285 +0x10a
k8s.io/kubernetes/test/e2e/network.glob..func20.3()
	test/e2e/network/loadbalancer.go:1312 +0x37f
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/node/init/init.go:32
Nov 26 12:38:22.773: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1260
Nov 26 12:38:22.816: INFO: 
Output of kubectl describe svc:

Nov 26 12:38:22.817: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-1787 describe svc --namespace=esipp-1787'
Nov 26 12:38:23.408: INFO: stderr: ""
Nov 26 12:38:23.408: INFO: stdout: "Name:              external-local-lb\nNamespace:         esipp-1787\nLabels:            testid=external-local-lb-40702d35-dff8-4b18-8ad7-faa70461880e\nAnnotations:       <none>\nSelector:          testid=external-local-lb-40702d35-dff8-4b18-8ad7-faa70461880e\nType:              ClusterIP\nIP Family Policy:  SingleStack\nIP Families:       IPv4\nIP:                10.0.125.19\nIPs:               10.0.125.19\nPort:              <unset>  80/TCP\nTargetPort:        80/TCP\nEndpoints:         10.64.0.59:80\nSession Affinity:  None\nEvents:\n  Type    Reason                Age    From                Message\n  ----    ------                ----   ----                -------\n  Normal  EnsuringLoadBalancer  3m56s  service-controller  Ensuring load balancer\n  Normal  EnsuredLoadBalancer   3m19s  service-controller  Ensured load balancer\n  Normal  DeletingLoadBalancer  91s    service-controller  Deleting load balancer\n  Normal  DeletedLoadBalancer   61s    service-controller  Deleted load balancer\n\n\nName:                     node-port-service\nNamespace:                esipp-1787\nLabels:                   <none>\nAnnotations:              <none>\nSelector:                 selector-b98ebd5a-a1f4-42fa-bf61-8ea4c2f541cd=true\nType:                     NodePort\nIP Family Policy:         SingleStack\nIP Families:              IPv4\nIP:                       10.0.190.103\nIPs:                      10.0.190.103\nPort:                     http  80/TCP\nTargetPort:               8083/TCP\nNodePort:                 http  30939/TCP\nEndpoints:                <none>\nPort:                     udp  90/UDP\nTargetPort:               8081/UDP\nNodePort:                 udp  30257/UDP\nEndpoints:                <none>\nSession Affinity:         None\nExternal Traffic Policy:  Cluster\nEvents:                   <none>\n\n\nName:                     session-affinity-service\nNamespace:                esipp-1787\nLabels:                   <none>\nAnnotations:              <none>\nSelector:                 selector-b98ebd5a-a1f4-42fa-bf61-8ea4c2f541cd=true\nType:                     NodePort\nIP Family Policy:         SingleStack\nIP Families:              IPv4\nIP:                       10.0.149.167\nIPs:                      10.0.149.167\nPort:                     http  80/TCP\nTargetPort:               8083/TCP\nNodePort:                 http  32689/TCP\nEndpoints:                <none>\nPort:                     udp  90/UDP\nTargetPort:               8081/UDP\nNodePort:                 udp  32263/UDP\nEndpoints:                <none>\nSession Affinity:         ClientIP\nExternal Traffic Policy:  Cluster\nEvents:                   <none>\n"
Nov 26 12:38:23.408: INFO: Name:              external-local-lb
Namespace:         esipp-1787
Labels:            testid=external-local-lb-40702d35-dff8-4b18-8ad7-faa70461880e
Annotations:       <none>
Selector:          testid=external-local-lb-40702d35-dff8-4b18-8ad7-faa70461880e
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.0.125.19
IPs:               10.0.125.19
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.64.0.59:80
Session Affinity:  None
Events:
  Type    Reason                Age    From                Message
  ----    ------                ----   ----                -------
  Normal  EnsuringLoadBalancer  3m56s  service-controller  Ensuring load balancer
  Normal  EnsuredLoadBalancer   3m19s  service-controller  Ensured load balancer
  Normal  DeletingLoadBalancer  91s    service-controller  Deleting load balancer
  Normal  DeletedLoadBalancer   61s    service-controller  Deleted load balancer


Name:                     node-port-service
Namespace:                esipp-1787
Labels:                   <none>
Annotations:              <none>
Selector:                 selector-b98ebd5a-a1f4-42fa-bf61-8ea4c2f541cd=true
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.190.103
IPs:                      10.0.190.103
Port:                     http  80/TCP
TargetPort:               8083/TCP
NodePort:                 http  30939/TCP
Endpoints:                <none>
Port:                     udp  90/UDP
TargetPort:               8081/UDP
NodePort:                 udp  30257/UDP
Endpoints:                <none>
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>


Name:                     session-affinity-service
Namespace:                esipp-1787
Labels:                   <none>
Annotations:              <none>
Selector:                 selector-b98ebd5a-a1f4-42fa-bf61-8ea4c2f541cd=true
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.149.167
IPs:                      10.0.149.167
Port:                     http  80/TCP
TargetPort:               8083/TCP
NodePort:                 http  32689/TCP
Endpoints:                <none>
Port:                     udp  90/UDP
TargetPort:               8081/UDP
NodePort:                 udp  32263/UDP
Endpoints:                <none>
Session Affinity:         ClientIP
External Traffic Policy:  Cluster
Events:                   <none>

[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:38:23.408
STEP: Collecting events from namespace "esipp-1787". 11/26/22 12:38:23.408
STEP: Found 36 events. 11/26/22 12:38:23.451
Nov 26 12:38:23.451: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for external-local-lb-8s8kb: { } Scheduled: Successfully assigned esipp-1787/external-local-lb-8s8kb to bootstrap-e2e-minion-group-8pmq
Nov 26 12:38:23.451: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-0: { } Scheduled: Successfully assigned esipp-1787/netserver-0 to bootstrap-e2e-minion-group-8pmq
Nov 26 12:38:23.451: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-1: { } Scheduled: Successfully assigned esipp-1787/netserver-1 to bootstrap-e2e-minion-group-dldd
Nov 26 12:38:23.451: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-2: { } Scheduled: Successfully assigned esipp-1787/netserver-2 to bootstrap-e2e-minion-group-tvmd
Nov 26 12:38:23.451: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for test-container-pod: { } Scheduled: Successfully assigned esipp-1787/test-container-pod to bootstrap-e2e-minion-group-tvmd
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:34:27 +0000 UTC - event for external-local-lb: {service-controller } EnsuringLoadBalancer: Ensuring load balancer
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:35:04 +0000 UTC - event for external-local-lb: {service-controller } EnsuredLoadBalancer: Ensured load balancer
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:35:06 +0000 UTC - event for external-local-lb: {replication-controller } SuccessfulCreate: Created pod: external-local-lb-8s8kb
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:35:21 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-n4464" : failed to sync configmap cache: timed out waiting for the condition
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:35:23 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} Started: Started container netexec
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:35:23 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} Created: Created container netexec
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:35:23 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:36:52 +0000 UTC - event for external-local-lb: {service-controller } DeletingLoadBalancer: Deleting load balancer
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Killing: Stopping container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Created: Created container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Started: Started container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-khl84" : failed to sync configmap cache: timed out waiting for the condition
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:08 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:09 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:10 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Created: Created container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:10 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Started: Started container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:10 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:11 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Killing: Stopping container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:12 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:17 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} BackOff: Back-off restarting failed container webserver in pod netserver-1_esipp-1787(db7bb86c-6741-46cb-8d8a-9d1448b47336)
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:22 +0000 UTC - event for external-local-lb: {service-controller } DeletedLoadBalancer: Deleted load balancer
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:44 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:44 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} Killing: Stopping container netexec
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:47 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} Unhealthy: Readiness probe failed: Get "http://10.64.0.74:80/hostName": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:47 +0000 UTC - event for external-local-lb-8s8kb: {kubelet bootstrap-e2e-minion-group-8pmq} BackOff: Back-off restarting failed container netexec in pod external-local-lb-8s8kb_esipp-1787(b8d2f1bb-a76e-49e6-8f93-e0b77282a936)
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:51 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:51 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container webserver
Nov 26 12:38:23.451: INFO: At 2022-11-26 12:37:51 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container webserver
Nov 26 12:38:23.493: INFO: POD                      NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:38:23.493: INFO: external-local-lb-8s8kb  bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:35:18 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:38:00 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:38:00 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:35:18 +0000 UTC  }]
Nov 26 12:38:23.493: INFO: netserver-0              bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:07 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:27 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:27 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:07 +0000 UTC  }]
Nov 26 12:38:23.493: INFO: netserver-1              bootstrap-e2e-minion-group-dldd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:07 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:49 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:49 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:07 +0000 UTC  }]
Nov 26 12:38:23.493: INFO: netserver-2              bootstrap-e2e-minion-group-tvmd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:07 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:27 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:27 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:07 +0000 UTC  }]
Nov 26 12:38:23.493: INFO: test-container-pod       bootstrap-e2e-minion-group-tvmd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:50 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:51 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:51 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:37:50 +0000 UTC  }]
Nov 26 12:38:23.493: INFO: 
Nov 26 12:38:23.763: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:38:23.813: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 4299 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:36:48 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:36:48 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:36:48 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:36:48 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:36:48 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:38:23.814: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:38:23.857: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:38:23.906: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container kube-apiserver ready: true, restart count 0
Nov 26 12:38:23.906: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container kube-controller-manager ready: false, restart count 3
Nov 26 12:38:23.906: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container kube-scheduler ready: true, restart count 4
Nov 26 12:38:23.906: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container etcd-container ready: true, restart count 0
Nov 26 12:38:23.906: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-26 12:30:55 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container l7-lb-controller ready: true, restart count 3
Nov 26 12:38:23.906: INFO: metadata-proxy-v0.1-n9n2g started at 2022-11-26 12:31:27 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:38:23.906: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:38:23.906: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container etcd-container ready: true, restart count 2
Nov 26 12:38:23.906: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-26 12:30:38 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container konnectivity-server-container ready: true, restart count 0
Nov 26 12:38:23.906: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-26 12:30:55 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:23.906: INFO: 	Container kube-addon-manager ready: true, restart count 0
Nov 26 12:38:24.112: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 26 12:38:24.112: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:38:24.154: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 5036 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6873":"bootstrap-e2e-minion-group-8pmq","csi-mock-csi-mock-volumes-5290":"csi-mock-csi-mock-volumes-5290"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-26 12:36:22 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:37:18 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:37:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:36:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:36:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:36:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:36:22 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:37:36 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:37:36 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:37:36 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:37:36 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:38:24.155: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:38:24.202: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:38:24.257: INFO: kube-proxy-bootstrap-e2e-minion-group-8pmq started at 2022-11-26 12:31:18 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container kube-proxy ready: true, restart count 4
Nov 26 12:38:24.257: INFO: addon-reconcile-test-fcn2z started at 2022-11-26 12:37:07 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container addon-reconcile-test ready: true, restart count 1
Nov 26 12:38:24.257: INFO: ss-0 started at 2022-11-26 12:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container webserver ready: true, restart count 2
Nov 26 12:38:24.257: INFO: konnectivity-agent-zdgr9 started at 2022-11-26 12:31:30 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container konnectivity-agent ready: false, restart count 5
Nov 26 12:38:24.257: INFO: csi-mockplugin-0 started at 2022-11-26 12:32:33 +0000 UTC (0+4 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container busybox ready: true, restart count 3
Nov 26 12:38:24.257: INFO: 	Container csi-provisioner ready: true, restart count 3
Nov 26 12:38:24.257: INFO: 	Container driver-registrar ready: true, restart count 3
Nov 26 12:38:24.257: INFO: 	Container mock ready: true, restart count 3
Nov 26 12:38:24.257: INFO: csi-mockplugin-0 started at 2022-11-26 12:32:34 +0000 UTC (0+4 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container busybox ready: false, restart count 4
Nov 26 12:38:24.257: INFO: 	Container csi-provisioner ready: false, restart count 3
Nov 26 12:38:24.257: INFO: 	Container driver-registrar ready: false, restart count 3
Nov 26 12:38:24.257: INFO: 	Container mock ready: false, restart count 3
Nov 26 12:38:24.257: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:33:31 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container csi-attacher ready: false, restart count 2
Nov 26 12:38:24.257: INFO: 	Container csi-provisioner ready: false, restart count 2
Nov 26 12:38:24.257: INFO: 	Container csi-resizer ready: false, restart count 2
Nov 26 12:38:24.257: INFO: 	Container csi-snapshotter ready: false, restart count 2
Nov 26 12:38:24.257: INFO: 	Container hostpath ready: false, restart count 2
Nov 26 12:38:24.257: INFO: 	Container liveness-probe ready: false, restart count 2
Nov 26 12:38:24.257: INFO: 	Container node-driver-registrar ready: false, restart count 2
Nov 26 12:38:24.257: INFO: netserver-0 started at 2022-11-26 12:37:07 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container webserver ready: true, restart count 1
Nov 26 12:38:24.257: INFO: coredns-6d97d5ddb-9gsq9 started at 2022-11-26 12:31:36 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container coredns ready: false, restart count 5
Nov 26 12:38:24.257: INFO: metadata-proxy-v0.1-ctfqs started at 2022-11-26 12:31:19 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:38:24.257: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:38:24.257: INFO: external-local-lb-8s8kb started at 2022-11-26 12:35:18 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container netexec ready: true, restart count 2
Nov 26 12:38:24.257: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:37:07 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:38:24.257: INFO: 	Container csi-attacher ready: true, restart count 1
Nov 26 12:38:24.257: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 26 12:38:24.257: INFO: 	Container csi-resizer ready: true, restart count 1
Nov 26 12:38:24.257: INFO: 	Container csi-snapshotter ready: true, restart count 1
Nov 26 12:38:24.257: INFO: 	Container hostpath ready: true, restart count 1
Nov 26 12:38:24.257: INFO: 	Container liveness-probe ready: true, restart count 1
Nov 26 12:38:24.257: INFO: 	Container node-driver-registrar ready: true, restart count 1
Nov 26 12:38:24.529: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:38:24.529: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:38:24.571: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 5079 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-3254":"bootstrap-e2e-minion-group-dldd","csi-hostpath-multivolume-5987":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-1645":"bootstrap-e2e-minion-group-dldd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:34:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-26 12:36:23 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-26 12:38:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:36:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:36:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:36:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:36:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:34:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:34:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:34:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:34:52 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:38:24.572: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:38:24.616: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:38:24.675: INFO: kube-proxy-bootstrap-e2e-minion-group-dldd started at 2022-11-26 12:31:18 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container kube-proxy ready: false, restart count 5
Nov 26 12:38:24.675: INFO: metadata-proxy-v0.1-2tdgn started at 2022-11-26 12:31:18 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:38:24.675: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:38:24.675: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:37:07 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 26 12:38:24.675: INFO: 	Container csi-provisioner ready: true, restart count 3
Nov 26 12:38:24.675: INFO: 	Container csi-resizer ready: true, restart count 3
Nov 26 12:38:24.675: INFO: 	Container csi-snapshotter ready: true, restart count 3
Nov 26 12:38:24.675: INFO: 	Container hostpath ready: true, restart count 3
Nov 26 12:38:24.675: INFO: 	Container liveness-probe ready: true, restart count 3
Nov 26 12:38:24.675: INFO: 	Container node-driver-registrar ready: true, restart count 3
Nov 26 12:38:24.675: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:33:37 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container csi-attacher ready: false, restart count 4
Nov 26 12:38:24.675: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 26 12:38:24.675: INFO: 	Container csi-resizer ready: false, restart count 4
Nov 26 12:38:24.675: INFO: 	Container csi-snapshotter ready: false, restart count 4
Nov 26 12:38:24.675: INFO: 	Container hostpath ready: false, restart count 4
Nov 26 12:38:24.675: INFO: 	Container liveness-probe ready: false, restart count 4
Nov 26 12:38:24.675: INFO: 	Container node-driver-registrar ready: false, restart count 4
Nov 26 12:38:24.675: INFO: netserver-1 started at 2022-11-26 12:37:07 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container webserver ready: true, restart count 2
Nov 26 12:38:24.675: INFO: hostexec-bootstrap-e2e-minion-group-dldd-hcj6x started at 2022-11-26 12:37:07 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 26 12:38:24.675: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:32:33 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 26 12:38:24.675: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 26 12:38:24.675: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 26 12:38:24.675: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 26 12:38:24.675: INFO: 	Container hostpath ready: true, restart count 2
Nov 26 12:38:24.675: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 26 12:38:24.675: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 26 12:38:24.675: INFO: ss-2 started at 2022-11-26 12:34:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container webserver ready: false, restart count 1
Nov 26 12:38:24.675: INFO: coredns-6d97d5ddb-cdh6x started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container coredns ready: true, restart count 5
Nov 26 12:38:24.675: INFO: l7-default-backend-8549d69d99-sxw6c started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container default-http-backend ready: true, restart count 0
Nov 26 12:38:24.675: INFO: konnectivity-agent-h9xml started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container konnectivity-agent ready: false, restart count 4
Nov 26 12:38:24.675: INFO: csi-mockplugin-attacher-0 started at 2022-11-26 12:32:33 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container csi-attacher ready: true, restart count 0
Nov 26 12:38:24.675: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:33:31 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container csi-attacher ready: false, restart count 5
Nov 26 12:38:24.675: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 26 12:38:24.675: INFO: 	Container csi-resizer ready: false, restart count 5
Nov 26 12:38:24.675: INFO: 	Container csi-snapshotter ready: false, restart count 5
Nov 26 12:38:24.675: INFO: 	Container hostpath ready: false, restart count 5
Nov 26 12:38:24.675: INFO: 	Container liveness-probe ready: false, restart count 5
Nov 26 12:38:24.675: INFO: 	Container node-driver-registrar ready: false, restart count 5
Nov 26 12:38:24.675: INFO: kube-dns-autoscaler-5f6455f985-xgvqj started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container autoscaler ready: false, restart count 4
Nov 26 12:38:24.675: INFO: volume-snapshot-controller-0 started at 2022-11-26 12:31:29 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container volume-snapshot-controller ready: true, restart count 2
Nov 26 12:38:24.675: INFO: csi-mockplugin-0 started at 2022-11-26 12:32:33 +0000 UTC (0+3 container statuses recorded)
Nov 26 12:38:24.675: INFO: 	Container csi-provisioner ready: true, restart count 0
Nov 26 12:38:24.675: INFO: 	Container driver-registrar ready: true, restart count 0
Nov 26 12:38:24.675: INFO: 	Container mock ready: true, restart count 0
Nov 26 12:38:24.922: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-dldd
Nov 26 12:38:24.922: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:38:24.965: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 5007 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-7061":"bootstrap-e2e-minion-group-tvmd","csi-mock-csi-mock-volumes-7960":"bootstrap-e2e-minion-group-tvmd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-26 12:36:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:37:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:37:49 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:36:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:36:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:36:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:36:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:36:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:37:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:37:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:37:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:37:49 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^1e6ff3f6-6d87-11ed-aea4-7ebd82c03ead],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^16d52ff8-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},},Config:nil,},}
Nov 26 12:38:24.966: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:38:25.023: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:38:25.132: INFO: pod-secrets-1f684501-a0a3-4930-8f59-c3dd29d2d355 started at 2022-11-26 12:35:27 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 26 12:38:25.132: INFO: nfs-server started at 2022-11-26 12:34:27 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container nfs-server ready: true, restart count 4
Nov 26 12:38:25.132: INFO: pod-secrets-e9c783f0-3356-41e8-a63e-f2a576a4b3ae started at 2022-11-26 12:35:32 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 26 12:38:25.132: INFO: netserver-2 started at 2022-11-26 12:37:07 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container webserver ready: true, restart count 0
Nov 26 12:38:25.132: INFO: addon-reconcile-test-qx4k9 started at 2022-11-26 12:37:07 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container addon-reconcile-test ready: true, restart count 0
Nov 26 12:38:25.132: INFO: kube-proxy-bootstrap-e2e-minion-group-tvmd started at 2022-11-26 12:31:20 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container kube-proxy ready: false, restart count 4
Nov 26 12:38:25.132: INFO: metadata-proxy-v0.1-qcjv9 started at 2022-11-26 12:31:20 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 26 12:38:25.132: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 26 12:38:25.132: INFO: konnectivity-agent-b72jm started at 2022-11-26 12:31:39 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container konnectivity-agent ready: true, restart count 4
Nov 26 12:38:25.132: INFO: csi-hostpathplugin-0 started at 2022-11-26 12:33:31 +0000 UTC (0+7 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container csi-attacher ready: true, restart count 1
Nov 26 12:38:25.132: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 26 12:38:25.132: INFO: 	Container csi-resizer ready: true, restart count 1
Nov 26 12:38:25.132: INFO: 	Container csi-snapshotter ready: true, restart count 1
Nov 26 12:38:25.132: INFO: 	Container hostpath ready: true, restart count 1
Nov 26 12:38:25.132: INFO: 	Container liveness-probe ready: true, restart count 1
Nov 26 12:38:25.132: INFO: 	Container node-driver-registrar ready: true, restart count 1
Nov 26 12:38:25.132: INFO: test-container-pod started at 2022-11-26 12:37:50 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container webserver ready: true, restart count 0
Nov 26 12:38:25.132: INFO: csi-mockplugin-0 started at 2022-11-26 12:37:23 +0000 UTC (0+3 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container csi-provisioner ready: true, restart count 0
Nov 26 12:38:25.132: INFO: 	Container driver-registrar ready: true, restart count 0
Nov 26 12:38:25.132: INFO: 	Container mock ready: true, restart count 0
Nov 26 12:38:25.132: INFO: pvc-volume-tester-cpjl6 started at 2022-11-26 12:37:31 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container volume-tester ready: false, restart count 0
Nov 26 12:38:25.132: INFO: pvc-volume-tester-p9ctz started at 2022-11-26 12:37:44 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container volume-tester ready: false, restart count 0
Nov 26 12:38:25.132: INFO: hostexec-bootstrap-e2e-minion-group-tvmd-8npsp started at 2022-11-26 12:37:22 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container agnhost-container ready: true, restart count 0
Nov 26 12:38:25.132: INFO: csi-mockplugin-attacher-0 started at 2022-11-26 12:37:23 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container csi-attacher ready: true, restart count 0
Nov 26 12:38:25.132: INFO: metrics-server-v0.5.2-867b8754b9-4kvml started at 2022-11-26 12:31:58 +0000 UTC (0+2 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container metrics-server ready: false, restart count 4
Nov 26 12:38:25.132: INFO: 	Container metrics-server-nanny ready: false, restart count 5
Nov 26 12:38:25.132: INFO: pod-secrets-42edbd47-ddd5-4bb5-9631-c2eaa23dbda2 started at 2022-11-26 12:37:19 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 26 12:38:25.132: INFO: back-off-cap started at 2022-11-26 12:34:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container back-off-cap ready: false, restart count 5
Nov 26 12:38:25.132: INFO: ss-1 started at 2022-11-26 12:34:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container webserver ready: true, restart count 3
Nov 26 12:38:25.132: INFO: mutability-test-hjlrj started at 2022-11-26 12:34:28 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container netexec ready: true, restart count 3
Nov 26 12:38:25.132: INFO: reallocate-nodeport-test-6lpqm started at 2022-11-26 12:35:34 +0000 UTC (0+1 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container netexec ready: true, restart count 3
Nov 26 12:38:25.132: INFO: csi-mockplugin-0 started at 2022-11-26 12:32:32 +0000 UTC (0+4 container statuses recorded)
Nov 26 12:38:25.132: INFO: 	Container busybox ready: false, restart count 2
Nov 26 12:38:25.132: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 26 12:38:25.132: INFO: 	Container driver-registrar ready: false, restart count 3
Nov 26 12:38:25.132: INFO: 	Container mock ready: false, restart count 3
Nov 26 12:38:25.372: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-tvmd
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  tear down framework | framework.go:193
STEP: Destroying namespace "esipp-1787" for this suite. 11/26/22 12:38:25.372

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-network] LoadBalancers ESIPP [Slow] should work for type=NodePort 3m31s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sLoadBalancers\sESIPP\s\[Slow\]\sshould\swork\sfor\stype\=NodePort$'

test/e2e/framework/network/utils.go:834
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0006f2460, 0x3c?)
	test/e2e/framework/network/utils.go:834 +0x545
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001112000, {0x0, 0x0, 0x0?})
	test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func20.4()
	test/e2e/network/loadbalancer.go:1332 +0x145
				[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:40:48.431
Nov 26 12:40:48.431: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename esipp 11/26/22 12:40:48.433
Nov 26 12:40:48.472: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:50.511: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:52.512: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:54.512: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:56.511: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:40:58.512: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:00.511: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:02.512: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:04.511: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:06.511: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:08.512: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:10.511: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:12.512: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:14.512: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
Nov 26 12:41:16.511: INFO: Unexpected error while creating namespace: Post "https://34.145.15.180/api/v1/namespaces": dial tcp 34.145.15.180:443: connect: connection refused
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:42:52.927
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:42:53.006
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1250
[It] should work for type=NodePort
  test/e2e/network/loadbalancer.go:1314
STEP: creating a service esipp-7662/external-local-nodeport with type=NodePort and ExternalTrafficPolicy=Local 11/26/22 12:42:56.976
STEP: creating a pod to be part of the service external-local-nodeport 11/26/22 12:42:57.034
Nov 26 12:42:57.077: INFO: Waiting up to 2m0s for 1 pods to be created
Nov 26 12:42:57.117: INFO: Found all 1 pods
Nov 26 12:42:57.118: INFO: Waiting up to 2m0s for 1 pods to be running and ready: [external-local-nodeport-czbm4]
Nov 26 12:42:57.118: INFO: Waiting up to 2m0s for pod "external-local-nodeport-czbm4" in namespace "esipp-7662" to be "running and ready"
Nov 26 12:42:57.166: INFO: Pod "external-local-nodeport-czbm4": Phase="Pending", Reason="", readiness=false. Elapsed: 48.733464ms
Nov 26 12:42:57.166: INFO: Error evaluating pod condition running and ready: want pod 'external-local-nodeport-czbm4' on 'bootstrap-e2e-minion-group-dldd' to be 'Running' but was 'Pending'
Nov 26 12:42:59.214: INFO: Pod "external-local-nodeport-czbm4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.096568475s
Nov 26 12:42:59.214: INFO: Error evaluating pod condition running and ready: want pod 'external-local-nodeport-czbm4' on 'bootstrap-e2e-minion-group-dldd' to be 'Running' but was 'Pending'
Nov 26 12:43:01.311: INFO: Pod "external-local-nodeport-czbm4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.193534817s
Nov 26 12:43:01.311: INFO: Error evaluating pod condition running and ready: want pod 'external-local-nodeport-czbm4' on 'bootstrap-e2e-minion-group-dldd' to be 'Running' but was 'Pending'
Nov 26 12:43:03.217: INFO: Pod "external-local-nodeport-czbm4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.099136592s
Nov 26 12:43:03.217: INFO: Error evaluating pod condition running and ready: want pod 'external-local-nodeport-czbm4' on 'bootstrap-e2e-minion-group-dldd' to be 'Running' but was 'Pending'
Nov 26 12:43:05.416: INFO: Pod "external-local-nodeport-czbm4": Phase="Running", Reason="", readiness=true. Elapsed: 8.29839197s
Nov 26 12:43:05.416: INFO: Pod "external-local-nodeport-czbm4" satisfied condition "running and ready"
Nov 26 12:43:05.416: INFO: Wanted all 1 pods to be running and ready. Result: true. Pods: [external-local-nodeport-czbm4]
STEP: Performing setup for networking test in namespace esipp-7662 11/26/22 12:43:06.631
STEP: creating a selector 11/26/22 12:43:06.631
STEP: Creating the service pods in kubernetes 11/26/22 12:43:06.631
Nov 26 12:43:06.631: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
Nov 26 12:43:06.947: INFO: Waiting up to 5m0s for pod "netserver-0" in namespace "esipp-7662" to be "running and ready"
Nov 26 12:43:07.004: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 56.526131ms
Nov 26 12:43:07.004: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:43:09.054: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 2.107383114s
Nov 26 12:43:09.055: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:43:11.067: INFO: Pod "netserver-0": Phase="Pending", Reason="", readiness=false. Elapsed: 4.120300489s
Nov 26 12:43:11.067: INFO: The phase of Pod netserver-0 is Pending, waiting for it to be Running (with Ready = true)
Nov 26 12:43:13.065: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 6.117524295s
Nov 26 12:43:13.065: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:15.062: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 8.114404782s
Nov 26 12:43:15.062: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:17.101: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 10.153537339s
Nov 26 12:43:17.101: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:19.075: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 12.127778729s
Nov 26 12:43:19.075: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:21.135: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 14.187965242s
Nov 26 12:43:21.135: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:23.084: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 16.136536567s
Nov 26 12:43:23.084: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:25.148: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 18.200907437s
Nov 26 12:43:25.148: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:27.070: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=false. Elapsed: 20.122702273s
Nov 26 12:43:27.070: INFO: The phase of Pod netserver-0 is Running (Ready = false)
Nov 26 12:43:29.061: INFO: Pod "netserver-0": Phase="Running", Reason="", readiness=true. Elapsed: 22.114168885s
Nov 26 12:43:29.061: INFO: The phase of Pod netserver-0 is Running (Ready = true)
Nov 26 12:43:29.061: INFO: Pod "netserver-0" satisfied condition "running and ready"
Nov 26 12:43:29.161: INFO: Waiting up to 5m0s for pod "netserver-1" in namespace "esipp-7662" to be "running and ready"
Nov 26 12:43:29.267: INFO: Pod "netserver-1": Phase="Running", Reason="", readiness=true. Elapsed: 106.27985ms
Nov 26 12:43:29.267: INFO: The phase of Pod netserver-1 is Running (Ready = true)
Nov 26 12:43:29.267: INFO: Pod "netserver-1" satisfied condition "running and ready"
Nov 26 12:43:29.333: INFO: Waiting up to 5m0s for pod "netserver-2" in namespace "esipp-7662" to be "running and ready"
Nov 26 12:43:29.397: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=false. Elapsed: 63.937047ms
Nov 26 12:43:29.397: INFO: The phase of Pod netserver-2 is Running (Ready = false)
Nov 26 12:43:31.472: INFO: Pod "netserver-2": Phase="Running", Reason="", readiness=true. Elapsed: 2.138388959s
Nov 26 12:43:31.472: INFO: The phase of Pod netserver-2 is Running (Ready = true)
Nov 26 12:43:31.472: INFO: Pod "netserver-2" satisfied condition "running and ready"
STEP: Creating test pods 11/26/22 12:43:31.56
Nov 26 12:43:31.732: INFO: Waiting up to 5m0s for pod "test-container-pod" in namespace "esipp-7662" to be "running"
Nov 26 12:43:31.849: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 116.55488ms
Nov 26 12:43:33.892: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 2.159622062s
Nov 26 12:43:35.911: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 4.178701083s
Nov 26 12:43:37.894: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 6.161845386s
Nov 26 12:43:39.923: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 8.190931997s
Nov 26 12:43:41.933: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 10.200724484s
Nov 26 12:43:43.915: INFO: Pod "test-container-pod": Phase="Pending", Reason="", readiness=false. Elapsed: 12.183314259s
Nov 26 12:43:45.925: INFO: Pod "test-container-pod": Phase="Running", Reason="", readiness=true. Elapsed: 14.19349756s
Nov 26 12:43:45.926: INFO: Pod "test-container-pod" satisfied condition "running"
Nov 26 12:43:45.984: INFO: Setting MaxTries for pod polling to 39 for networking test based on endpoint count 3
STEP: Getting node addresses 11/26/22 12:43:45.984
Nov 26 12:43:45.984: INFO: Waiting up to 10m0s for all (but 0) nodes to be schedulable
STEP: Creating the service on top of the pods in kubernetes 11/26/22 12:43:46.142
Nov 26 12:43:46.481: INFO: Service node-port-service in namespace esipp-7662 found.
Nov 26 12:43:46.994: INFO: Service session-affinity-service in namespace esipp-7662 found.
STEP: Waiting for NodePort service to expose endpoint 11/26/22 12:43:47.218
Nov 26 12:43:48.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:49.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:50.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:51.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:52.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:53.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:54.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:55.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:56.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:57.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:58.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:43:59.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:00.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:01.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:02.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:03.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:04.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:05.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:06.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:07.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:08.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:09.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:10.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:11.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:12.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:13.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:14.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:15.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:16.219: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:17.218: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:17.300: INFO: Waiting for amount of service:node-port-service endpoints to be 3
Nov 26 12:44:17.387: INFO: Unexpected error: failed to validate endpoints for service node-port-service in namespace: esipp-7662: 
    <*errors.errorString | 0xc0001fda10>: {
        s: "timed out waiting for the condition",
    }
Nov 26 12:44:17.387: FAIL: failed to validate endpoints for service node-port-service in namespace: esipp-7662: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/network.(*NetworkingTestConfig).setup(0xc0006f2460, 0x3c?)
	test/e2e/framework/network/utils.go:834 +0x545
k8s.io/kubernetes/test/e2e/framework/network.NewNetworkingTestConfig(0xc001112000, {0x0, 0x0, 0x0?})
	test/e2e/framework/network/utils.go:131 +0x125
k8s.io/kubernetes/test/e2e/network.glob..func20.4()
	test/e2e/network/loadbalancer.go:1332 +0x145
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/node/init/init.go:32
Nov 26 12:44:17.617: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1260
Nov 26 12:44:17.692: INFO: 
Output of kubectl describe svc:

Nov 26 12:44:17.692: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-7662 describe svc --namespace=esipp-7662'
Nov 26 12:44:18.281: INFO: stderr: ""
Nov 26 12:44:18.281: INFO: stdout: "Name:                     node-port-service\nNamespace:                esipp-7662\nLabels:                   <none>\nAnnotations:              <none>\nSelector:                 selector-49d1b2bb-60b2-4f87-b9aa-713c9415dca3=true\nType:                     NodePort\nIP Family Policy:         SingleStack\nIP Families:              IPv4\nIP:                       10.0.200.38\nIPs:                      10.0.200.38\nPort:                     http  80/TCP\nTargetPort:               8083/TCP\nNodePort:                 http  30157/TCP\nEndpoints:                10.64.1.131:8083,10.64.2.147:8083\nPort:                     udp  90/UDP\nTargetPort:               8081/UDP\nNodePort:                 udp  31642/UDP\nEndpoints:                10.64.1.131:8081,10.64.2.147:8081\nSession Affinity:         None\nExternal Traffic Policy:  Cluster\nEvents:                   <none>\n\n\nName:                     session-affinity-service\nNamespace:                esipp-7662\nLabels:                   <none>\nAnnotations:              <none>\nSelector:                 selector-49d1b2bb-60b2-4f87-b9aa-713c9415dca3=true\nType:                     NodePort\nIP Family Policy:         SingleStack\nIP Families:              IPv4\nIP:                       10.0.72.103\nIPs:                      10.0.72.103\nPort:                     http  80/TCP\nTargetPort:               8083/TCP\nNodePort:                 http  31379/TCP\nEndpoints:                10.64.1.131:8083,10.64.2.147:8083\nPort:                     udp  90/UDP\nTargetPort:               8081/UDP\nNodePort:                 udp  30262/UDP\nEndpoints:                10.64.1.131:8081,10.64.2.147:8081\nSession Affinity:         ClientIP\nExternal Traffic Policy:  Cluster\nEvents:                   <none>\n"
Nov 26 12:44:18.281: INFO: Name:                     node-port-service
Namespace:                esipp-7662
Labels:                   <none>
Annotations:              <none>
Selector:                 selector-49d1b2bb-60b2-4f87-b9aa-713c9415dca3=true
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.200.38
IPs:                      10.0.200.38
Port:                     http  80/TCP
TargetPort:               8083/TCP
NodePort:                 http  30157/TCP
Endpoints:                10.64.1.131:8083,10.64.2.147:8083
Port:                     udp  90/UDP
TargetPort:               8081/UDP
NodePort:                 udp  31642/UDP
Endpoints:                10.64.1.131:8081,10.64.2.147:8081
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>


Name:                     session-affinity-service
Namespace:                esipp-7662
Labels:                   <none>
Annotations:              <none>
Selector:                 selector-49d1b2bb-60b2-4f87-b9aa-713c9415dca3=true
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.0.72.103
IPs:                      10.0.72.103
Port:                     http  80/TCP
TargetPort:               8083/TCP
NodePort:                 http  31379/TCP
Endpoints:                10.64.1.131:8083,10.64.2.147:8083
Port:                     udp  90/UDP
TargetPort:               8081/UDP
NodePort:                 udp  30262/UDP
Endpoints:                10.64.1.131:8081,10.64.2.147:8081
Session Affinity:         ClientIP
External Traffic Policy:  Cluster
Events:                   <none>

[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/26/22 12:44:18.281
STEP: Collecting events from namespace "esipp-7662". 11/26/22 12:44:18.282
STEP: Found 28 events. 11/26/22 12:44:18.345
Nov 26 12:44:18.346: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for external-local-nodeport-czbm4: { } Scheduled: Successfully assigned esipp-7662/external-local-nodeport-czbm4 to bootstrap-e2e-minion-group-dldd
Nov 26 12:44:18.346: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-0: { } Scheduled: Successfully assigned esipp-7662/netserver-0 to bootstrap-e2e-minion-group-8pmq
Nov 26 12:44:18.346: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-1: { } Scheduled: Successfully assigned esipp-7662/netserver-1 to bootstrap-e2e-minion-group-dldd
Nov 26 12:44:18.346: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for netserver-2: { } Scheduled: Successfully assigned esipp-7662/netserver-2 to bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:18.346: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for test-container-pod: { } Scheduled: Successfully assigned esipp-7662/test-container-pod to bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:42:57 +0000 UTC - event for external-local-nodeport: {replication-controller } SuccessfulCreate: Created pod: external-local-nodeport-czbm4
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:42:58 +0000 UTC - event for external-local-nodeport-czbm4: {kubelet bootstrap-e2e-minion-group-dldd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:42:58 +0000 UTC - event for external-local-nodeport-czbm4: {kubelet bootstrap-e2e-minion-group-dldd} Created: Created container netexec
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:42:58 +0000 UTC - event for external-local-nodeport-czbm4: {kubelet bootstrap-e2e-minion-group-dldd} Started: Started container netexec
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:01 +0000 UTC - event for external-local-nodeport-czbm4: {kubelet bootstrap-e2e-minion-group-dldd} Killing: Stopping container netexec
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:02 +0000 UTC - event for external-local-nodeport-czbm4: {kubelet bootstrap-e2e-minion-group-dldd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:09 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Created: Created container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:09 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Started: Started container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:09 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:09 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Created: Created container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:09 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:10 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} Killing: Stopping container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:10 +0000 UTC - event for netserver-1: {kubelet bootstrap-e2e-minion-group-dldd} Started: Started container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:10 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:10 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:10 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:11 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:11 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} Killing: Stopping container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:12 +0000 UTC - event for netserver-2: {kubelet bootstrap-e2e-minion-group-tvmd} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:30 +0000 UTC - event for netserver-0: {kubelet bootstrap-e2e-minion-group-8pmq} BackOff: Back-off restarting failed container webserver in pod netserver-0_esipp-7662(09fc453c-c964-4ca3-9c3f-1feef479c83a)
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:34 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-tvmd} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:34 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-tvmd} Created: Created container webserver
Nov 26 12:44:18.346: INFO: At 2022-11-26 12:43:35 +0000 UTC - event for test-container-pod: {kubelet bootstrap-e2e-minion-group-tvmd} Started: Started container webserver
Nov 26 12:44:18.396: INFO: POD                            NODE                             PHASE    GRACE  CONDITIONS
Nov 26 12:44:18.396: INFO: external-local-nodeport-czbm4  bootstrap-e2e-minion-group-dldd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:42:57 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:07 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:07 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:42:57 +0000 UTC  }]
Nov 26 12:44:18.397: INFO: netserver-0                    bootstrap-e2e-minion-group-8pmq  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:06 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:29 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:29 +0000 UTC ContainersNotReady containers with unready status: [webserver]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:06 +0000 UTC  }]
Nov 26 12:44:18.397: INFO: netserver-1                    bootstrap-e2e-minion-group-dldd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:06 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:28 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:28 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:06 +0000 UTC  }]
Nov 26 12:44:18.397: INFO: netserver-2                    bootstrap-e2e-minion-group-tvmd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:06 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:30 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:30 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:06 +0000 UTC  }]
Nov 26 12:44:18.397: INFO: test-container-pod             bootstrap-e2e-minion-group-tvmd  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:31 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:35 +0000 UTC  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:35 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-26 12:43:31 +0000 UTC  }]
Nov 26 12:44:18.397: INFO: 
Nov 26 12:44:18.488: INFO: Unable to fetch esipp-7662/external-local-nodeport-czbm4/netexec logs: an error on the server ("unknown") has prevented the request from succeeding (get pods external-local-nodeport-czbm4)
Nov 26 12:44:18.637: INFO: Unable to fetch esipp-7662/netserver-0/webserver logs: an error on the server ("unknown") has prevented the request from succeeding (get pods netserver-0)
Nov 26 12:44:18.794: INFO: Unable to fetch esipp-7662/netserver-1/webserver logs: an error on the server ("unknown") has prevented the request from succeeding (get pods netserver-1)
Nov 26 12:44:18.895: INFO: Unable to fetch esipp-7662/netserver-2/webserver logs: an error on the server ("unknown") has prevented the request from succeeding (get pods netserver-2)
Nov 26 12:44:19.071: INFO: Unable to fetch esipp-7662/test-container-pod/webserver logs: an error on the server ("unknown") has prevented the request from succeeding (get pods test-container-pod)
Nov 26 12:44:19.134: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 26 12:44:19.194: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    ef85f9b6-3667-45c4-bcaf-7104effb72ee 6333 0 2022-11-26 12:31:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:41:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:41:55 +0000 UTC,LastTransitionTime:2022-11-26 12:31:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.15.180,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:3697f21d5ec13de6d09f5f6092dc1fc4,SystemUUID:3697f21d-5ec1-3de6-d09f-5f6092dc1fc4,BootID:7a2e7998-d36f-4ad5-b1dd-13717351a73c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:135160272,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:44:19.195: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 26 12:44:19.275: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 26 12:44:19.350: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 26 12:44:19.350: INFO: 
Logging node info for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:44:19.404: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-8pmq    dc029b50-cbc4-4cfd-95e5-f6b182f554c8 8818 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-8pmq kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-8pmq topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-2975":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-multivolume-5002":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-multivolume-7037":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-1783":"bootstrap-e2e-minion-group-8pmq","csi-hostpath-provisioning-3960":"bootstrap-e2e-minion-group-8pmq","csi-mock-csi-mock-volumes-3803":"bootstrap-e2e-minion-group-8pmq"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-26 12:41:23 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:44:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:44:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-8pmq,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:41:23 +0000 UTC,LastTransitionTime:2022-11-26 12:31:21 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:44:15 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:44:15 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:44:15 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:44:15 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.127.12.232,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-8pmq.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:0fc34b60d62c031416e8a739a6cc33a4,SystemUUID:0fc34b60-d62c-0314-16e8-a739a6cc33a4,BootID:a3919bbb-b299-42a1-b4bd-8464dea27a46,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-multivolume-5002^f6ec6f89-6d87-11ed-8904-52ce12031d16 kubernetes.io/csi/csi-hostpath-multivolume-5002^f7c6a89c-6d87-11ed-8904-52ce12031d16],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f7c6a89c-6d87-11ed-8904-52ce12031d16,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-multivolume-5002^f6ec6f89-6d87-11ed-8904-52ce12031d16,DevicePath:,},},Config:nil,},}
Nov 26 12:44:19.404: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-8pmq
Nov 26 12:44:19.488: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-8pmq
Nov 26 12:44:19.573: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-8pmq: error trying to reach service: No agent available
Nov 26 12:44:19.573: INFO: 
Logging node info for node bootstrap-e2e-minion-group-dldd
Nov 26 12:44:19.630: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-dldd    e63b8924-1ca1-46da-8ab8-ad46b9f4cbbb 8461 0 2022-11-26 12:31:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-dldd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-dldd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-7800":"bootstrap-e2e-minion-group-dldd","csi-hostpath-volumeio-6147":"bootstrap-e2e-minion-group-dldd","csi-mock-csi-mock-volumes-1645":"bootstrap-e2e-minion-group-dldd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kubelet Update v1 2022-11-26 12:31:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-26 12:41:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:43:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-26 12:43:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-dldd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:41:24 +0000 UTC,LastTransitionTime:2022-11-26 12:31:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:29 +0000 UTC,LastTransitionTime:2022-11-26 12:31:29 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:43:54 +0000 UTC,LastTransitionTime:2022-11-26 12:31:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.247.4.139,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-dldd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:5440de85ef702c56968ba89e1fecdd56,SystemUUID:5440de85-ef70-2c56-968b-a89e1fecdd56,BootID:1359ab86-be89-4d6e-935e-63921e91662c,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 26 12:44:19.631: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-dldd
Nov 26 12:44:19.742: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-dldd
Nov 26 12:44:19.857: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-dldd: error trying to reach service: No agent available
Nov 26 12:44:19.857: INFO: 
Logging node info for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:19.918: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tvmd    fa454c49-5ac0-4c12-82cb-503cc69df46e 8725 0 2022-11-26 12:31:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tvmd kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tvmd topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-multivolume-3438":"bootstrap-e2e-minion-group-tvmd","csi-hostpath-provisioning-7215":"bootstrap-e2e-minion-group-tvmd","csi-mock-csi-mock-volumes-7960":"bootstrap-e2e-minion-group-tvmd","csi-mock-csi-mock-volumes-9921":"bootstrap-e2e-minion-group-tvmd"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-26 12:31:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-26 12:31:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-26 12:41:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-26 12:43:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-26 12:44:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-e2e-gke-ubuntu-serial/us-west1-b/bootstrap-e2e-minion-group-tvmd,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-26 12:41:25 +0000 UTC,LastTransitionTime:2022-11-26 12:31:23 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-26 12:31:39 +0000 UTC,LastTransitionTime:2022-11-26 12:31:39 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-26 12:43:58 +0000 UTC,LastTransitionTime:2022-11-26 12:31:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:35.233.192.146,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tvmd.c.k8s-jkns-e2e-gke-ubuntu-serial.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:caae3b6d0610846d2b3037eb29faa25f,SystemUUID:caae3b6d-0610-846d-2b30-37eb29faa25f,BootID:3c08c0b1-4ef9-4b1d-93ec-05afbf41d615,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.50+70617042976dc1,KubeProxyVersion:v1.27.0-alpha.0.50+70617042976dc1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.50_70617042976dc1],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead kubernetes.io/csi/csi-mock-csi-mock-volumes-9921^dee9b0e7-6d87-11ed-8ad2-265b8ea29070],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-7960^50d158ac-6d87-11ed-aea4-7ebd82c03ead,DevicePath:,},},Config:nil,},}
Nov 26 12:44:19.918: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:20.006: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tvmd
Nov 26 12:44:20.168: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-tvmd: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-network] LoadBalancers ESIPP [Slow]
  tear down framework | framework.go:193
STEP: Destroying namespace "esipp-7662" for this suite. 11/26/22 12:44:20.168

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-network] LoadBalancers ESIPP [Slow] should work from pods 12m43s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sLoadBalancers\sESIPP\s\[Slow\]\sshould\swork\sfrom\spods$'

test/e2e/network/loadbalancer.go:1476
k8s.io/kubernetes/test/e2e/network.glob..func20.6()
	test/e2e/network/loadbalancer.go:1476 +0xabd
				[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/26/22 12:37:21.275
Nov 26 12:37:21.275: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename esipp 11/26/22 12:37:21.277
STEP: Waiting for a default service account to be provisioned in namespace 11/26/22 12:37:21.473
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/26/22 12:37:21.571
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-network] LoadBalancers ESIPP [Slow]
  test/e2e/network/loadbalancer.go:1250
[It] should work from pods
  test/e2e/network/loadbalancer.go:1422
STEP: creating a service esipp-3856/external-local-pods with type=LoadBalancer 11/26/22 12:37:21.898
STEP: setting ExternalTrafficPolicy=Local 11/26/22 12:37:21.898
STEP: waiting for loadbalancer for service esipp-3856/external-local-pods 11/26/22 12:37:22.095
Nov 26 12:37:22.096: INFO: Waiting up to 15m0s for service "external-local-pods" to have a LoadBalancer
STEP: creating a pod to be part of the service external-local-pods 11/26/22 12:39:44.208
Nov 26 12:39:44.294: INFO: Waiting up to 2m0s for 1 pods to be created
Nov 26 12:39:44.376: INFO: Found 0/1 pods - will retry
Nov 26 12:39:46.474: INFO: Found all 1 pods
Nov 26 12:39:46.474: INFO: Waiting up to 2m0s for 1 pods to be running and ready: [external-local-pods-7nqc8]
Nov 26 12:39:46.474: INFO: Waiting up to 2m0s for pod "external-local-pods-7nqc8" in namespace "esipp-3856" to be "running and ready"
Nov 26 12:39:46.544: INFO: Pod "external-local-pods-7nqc8": Phase="Running", Reason="", readiness=true. Elapsed: 69.817207ms
Nov 26 12:39:46.544: INFO: Pod "external-local-pods-7nqc8" satisfied condition "running and ready"
Nov 26 12:39:46.544: INFO: Wanted all 1 pods to be running and ready. Result: true. Pods: [external-local-pods-7nqc8]
STEP: waiting for loadbalancer for service esipp-3856/external-local-pods 11/26/22 12:39:46.544
Nov 26 12:39:46.544: INFO: Waiting up to 15m0s for service "external-local-pods" to have a LoadBalancer
STEP: Creating pause pod deployment to make sure, pausePods are in desired state 11/26/22 12:39:46.627
Nov 26 12:39:46.769: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:0, Replicas:0, UpdatedReplicas:0, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:0, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 26, 12, 39, 46, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 26, 12, 39, 46, 0, time.Local), Reason:"NewReplicaSetCreated", Message:"Created new replica set \"pause-pod-deployment-7c748f845d\""}}, CollisionCount:(*int32)(nil)}
Nov 26 12:39:48.833: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 26, 12, 39, 46, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 26, 12, 39, 46, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 26, 12, 39, 46, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 26, 12, 39, 46, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"pause-pod-deployment-7c748f845d\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 26 12:39:51.010: INFO: Waiting up to 5m0s curl 35.247.92.206:80/clientip
STEP: Hitting external lb 35.247.92.206 from pod pause-pod-deployment-7c748f845d-rtr6l on node bootstrap-e2e-minion-group-tvmd 11/26/22 12:39:51.078
Nov 26 12:39:51.078: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:39:51.637: INFO: rc: 1
Nov 26 12:39:51.637: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:39:53.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:39:54.518: INFO: rc: 7
Nov 26 12:39:54.518: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:39:55.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:39:56.640: INFO: rc: 7
Nov 26 12:39:56.640: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:39:57.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:39:58.338: INFO: rc: 7
Nov 26 12:39:58.338: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:39:59.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:00.407: INFO: rc: 7
Nov 26 12:40:00.407: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:40:01.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:02.461: INFO: rc: 7
Nov 26 12:40:02.461: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:40:03.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:04.294: INFO: rc: 7
Nov 26 12:40:04.294: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:40:05.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:06.394: INFO: rc: 7
Nov 26 12:40:06.395: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:40:07.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:08.396: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:08.396: INFO: stdout: "10.64.2.119:51206"
Nov 26 12:40:09.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:10.632: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:10.632: INFO: stdout: "10.64.2.119:51208"
Nov 26 12:40:11.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:12.514: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:12.515: INFO: stdout: "10.64.2.119:51210"
Nov 26 12:40:13.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:14.299: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:14.299: INFO: stdout: "10.64.2.119:57028"
Nov 26 12:40:15.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:16.387: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:16.387: INFO: stdout: "10.64.2.119:57030"
Nov 26 12:40:17.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:18.429: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:18.429: INFO: stdout: "10.64.2.119:57032"
Nov 26 12:40:19.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:20.390: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:20.390: INFO: stdout: "10.64.2.119:57034"
Nov 26 12:40:21.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:22.457: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:22.457: INFO: stdout: "10.64.2.119:57036"
Nov 26 12:40:23.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:24.352: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:24.352: INFO: stdout: "10.64.2.119:33866"
Nov 26 12:40:25.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:26.427: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:26.427: INFO: stdout: "10.64.2.119:33868"
Nov 26 12:40:27.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:28.329: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:28.329: INFO: stdout: "10.64.2.119:33870"
Nov 26 12:40:29.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:30.342: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:30.342: INFO: stdout: "10.64.2.119:33872"
Nov 26 12:40:31.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:32.390: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:32.390: INFO: stdout: "10.64.2.119:33874"
Nov 26 12:40:33.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:34.309: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:34.309: INFO: stdout: "10.64.2.119:54282"
Nov 26 12:40:35.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:36.500: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:36.500: INFO: stdout: "10.64.2.119:54284"
Nov 26 12:40:37.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:38.364: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:38.364: INFO: stdout: "10.64.2.119:54286"
Nov 26 12:40:39.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:40.428: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:40.428: INFO: stdout: "10.64.2.119:54288"
Nov 26 12:40:41.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:42.376: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:42.376: INFO: stdout: "10.64.2.119:54290"
Nov 26 12:40:43.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:44.310: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:44.310: INFO: stdout: "10.64.2.119:57796"
Nov 26 12:40:45.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:46.349: INFO: stderr: "+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip\n"
Nov 26 12:40:46.349: INFO: stdout: "10.64.2.119:57798"
Nov 26 12:40:47.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:47.779: INFO: rc: 1
Nov 26 12:40:47.779: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:40:49.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:49.749: INFO: rc: 1
Nov 26 12:40:49.749: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:40:51.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:51.747: INFO: rc: 1
Nov 26 12:40:51.747: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:40:53.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:53.748: INFO: rc: 1
Nov 26 12:40:53.748: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:40:55.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:55.753: INFO: rc: 1
Nov 26 12:40:55.753: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:40:57.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:57.745: INFO: rc: 1
Nov 26 12:40:57.745: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:40:59.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:40:59.745: INFO: rc: 1
Nov 26 12:40:59.745: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:01.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:01.748: INFO: rc: 1
Nov 26 12:41:01.748: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:03.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:03.746: INFO: rc: 1
Nov 26 12:41:03.746: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:05.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:05.749: INFO: rc: 1
Nov 26 12:41:05.749: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:07.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:07.748: INFO: rc: 1
Nov 26 12:41:07.748: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:09.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:09.752: INFO: rc: 1
Nov 26 12:41:09.752: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:11.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:11.750: INFO: rc: 1
Nov 26 12:41:11.750: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:13.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:13.749: INFO: rc: 1
Nov 26 12:41:13.749: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:15.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:15.750: INFO: rc: 1
Nov 26 12:41:15.750: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:17.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:17.747: INFO: rc: 1
Nov 26 12:41:17.747: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
The connection to the server 34.145.15.180 was refused - did you specify the right host or port?

error:
exit status 1, retry until timeout
Nov 26 12:41:19.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:22.097: INFO: rc: 1
Nov 26 12:41:22.098: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:23.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:23.994: INFO: rc: 1
Nov 26 12:41:23.994: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:25.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:25.980: INFO: rc: 1
Nov 26 12:41:25.980: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:27.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:27.995: INFO: rc: 1
Nov 26 12:41:27.995: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:29.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:30.031: INFO: rc: 1
Nov 26 12:41:30.031: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:31.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:31.982: INFO: rc: 1
Nov 26 12:41:31.982: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:33.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:33.980: INFO: rc: 1
Nov 26 12:41:33.980: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:35.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:36.043: INFO: rc: 1
Nov 26 12:41:36.043: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
error: unable to upgrade connection: container not found ("agnhost-pause")

error:
exit status 1, retry until timeout
Nov 26 12:41:37.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:38.150: INFO: rc: 7
Nov 26 12:41:38.150: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:39.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:40.215: INFO: rc: 7
Nov 26 12:41:40.215: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:41.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:42.154: INFO: rc: 7
Nov 26 12:41:42.154: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:43.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:44.146: INFO: rc: 7
Nov 26 12:41:44.146: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:45.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:46.148: INFO: rc: 7
Nov 26 12:41:46.148: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:47.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:48.154: INFO: rc: 7
Nov 26 12:41:48.154: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:49.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:50.196: INFO: rc: 7
Nov 26 12:41:50.196: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:51.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:52.147: INFO: rc: 7
Nov 26 12:41:52.147: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:53.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:54.154: INFO: rc: 7
Nov 26 12:41:54.154: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:55.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:56.164: INFO: rc: 7
Nov 26 12:41:56.164: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:57.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:41:58.151: INFO: rc: 7
Nov 26 12:41:58.151: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:41:59.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:00.195: INFO: rc: 7
Nov 26 12:42:00.195: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:01.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:02.148: INFO: rc: 7
Nov 26 12:42:02.148: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:03.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:04.152: INFO: rc: 7
Nov 26 12:42:04.152: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:05.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:06.164: INFO: rc: 7
Nov 26 12:42:06.164: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:07.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:08.165: INFO: rc: 7
Nov 26 12:42:08.165: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:09.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:10.202: INFO: rc: 7
Nov 26 12:42:10.202: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:11.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:12.156: INFO: rc: 7
Nov 26 12:42:12.156: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:13.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:14.204: INFO: rc: 7
Nov 26 12:42:14.204: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:15.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:16.169: INFO: rc: 7
Nov 26 12:42:16.169: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:17.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:18.174: INFO: rc: 7
Nov 26 12:42:18.174: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:19.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:20.208: INFO: rc: 7
Nov 26 12:42:20.208: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:21.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
------------------------------
Progress Report for Ginkgo Process #6
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should work from pods (Spec Runtime: 5m0.623s)
    test/e2e/network/loadbalancer.go:1422
    In [It] (Node Runtime: 5m0.001s)
      test/e2e/network/loadbalancer.go:1422
      At [By Step] Hitting external lb 35.247.92.206 from pod pause-pod-deployment-7c748f845d-rtr6l on node bootstrap-e2e-minion-group-tvmd (Step Runtime: 2m30.821s)
        test/e2e/network/loadbalancer.go:1466

      Spec Goroutine
      goroutine 956 [select]
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.ExecWithFullOutput({0xc000d60840?, 0x0?})
          test/e2e/framework/kubectl/builder.go:125
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.Exec(...)
          test/e2e/framework/kubectl/builder.go:107
        k8s.io/kubernetes/test/e2e/framework/kubectl.RunKubectl({0xc00450e7d0?, 0x1?}, {0xc004e91ad8?, 0x101010020?, 0x0?})
          test/e2e/framework/kubectl/builder.go:154
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmd(...)
          test/e2e/framework/pod/output/output.go:82
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6.3()
          test/e2e/network/loadbalancer.go:1468
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1({0x2742871, 0x0})
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:222
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext({0x7fe0bc8?, 0xc00012e000?}, 0xc00456f0c0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:235
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc00012e000}, 0xc004f8acc0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:662
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc00012e000}, 0xb0?, 0x2fd9d05?, 0x28?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc00012e000}, 0x0?, 0xc004e91d00?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc00141b080?, 0x78?, 0x0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6()
          test/e2e/network/loadbalancer.go:1467
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004506c00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:42:22.161: INFO: rc: 7
Nov 26 12:42:22.161: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:23.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:24.201: INFO: rc: 7
Nov 26 12:42:24.201: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:25.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:26.175: INFO: rc: 7
Nov 26 12:42:26.175: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:27.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:28.214: INFO: rc: 7
Nov 26 12:42:28.214: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:29.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:30.212: INFO: rc: 7
Nov 26 12:42:30.212: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:31.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:32.164: INFO: rc: 7
Nov 26 12:42:32.164: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:33.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:34.180: INFO: rc: 7
Nov 26 12:42:34.180: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:35.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:36.175: INFO: rc: 7
Nov 26 12:42:36.175: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
+ curl -q -s --connect-timeout 30 35.247.92.206:80/clientip
command terminated with exit code 7

error:
exit status 7, retry until timeout
Nov 26 12:42:37.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:37.970: INFO: rc: 1
Nov 26 12:42:37.971: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:39.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:40.004: INFO: rc: 1
Nov 26 12:42:40.004: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:41.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
------------------------------
Progress Report for Ginkgo Process #6
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should work from pods (Spec Runtime: 5m20.626s)
    test/e2e/network/loadbalancer.go:1422
    In [It] (Node Runtime: 5m20.003s)
      test/e2e/network/loadbalancer.go:1422
      At [By Step] Hitting external lb 35.247.92.206 from pod pause-pod-deployment-7c748f845d-rtr6l on node bootstrap-e2e-minion-group-tvmd (Step Runtime: 2m50.823s)
        test/e2e/network/loadbalancer.go:1466

      Spec Goroutine
      goroutine 956 [select]
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.ExecWithFullOutput({0xc004ee2b00?, 0x0?})
          test/e2e/framework/kubectl/builder.go:125
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.Exec(...)
          test/e2e/framework/kubectl/builder.go:107
        k8s.io/kubernetes/test/e2e/framework/kubectl.RunKubectl({0xc00450e7d0?, 0x1?}, {0xc004e91ad8?, 0x101010020?, 0x0?})
          test/e2e/framework/kubectl/builder.go:154
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmd(...)
          test/e2e/framework/pod/output/output.go:82
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6.3()
          test/e2e/network/loadbalancer.go:1468
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1({0x2742871, 0x0})
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:222
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext({0x7fe0bc8?, 0xc00012e000?}, 0xc00456f0c0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:235
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc00012e000}, 0xc004f8acc0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:662
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc00012e000}, 0xb0?, 0x2fd9d05?, 0x28?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc00012e000}, 0x0?, 0xc004e91d00?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc00141b080?, 0x78?, 0x0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6()
          test/e2e/network/loadbalancer.go:1467
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004506c00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:42:41.999: INFO: rc: 1
Nov 26 12:42:41.999: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:43.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:43.967: INFO: rc: 1
Nov 26 12:42:43.967: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:45.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:45.967: INFO: rc: 1
Nov 26 12:42:45.967: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:47.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:47.966: INFO: rc: 1
Nov 26 12:42:47.966: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:49.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:50.012: INFO: rc: 1
Nov 26 12:42:50.012: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:51.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:52.108: INFO: rc: 1
Nov 26 12:42:52.108: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:53.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:53.968: INFO: rc: 1
Nov 26 12:42:53.968: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:55.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:55.973: INFO: rc: 1
Nov 26 12:42:55.973: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:57.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:42:58.120: INFO: rc: 1
Nov 26 12:42:58.120: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:42:59.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:00.132: INFO: rc: 1
Nov 26 12:43:00.132: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:01.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
------------------------------
Progress Report for Ginkgo Process #6
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should work from pods (Spec Runtime: 5m40.629s)
    test/e2e/network/loadbalancer.go:1422
    In [It] (Node Runtime: 5m40.006s)
      test/e2e/network/loadbalancer.go:1422
      At [By Step] Hitting external lb 35.247.92.206 from pod pause-pod-deployment-7c748f845d-rtr6l on node bootstrap-e2e-minion-group-tvmd (Step Runtime: 3m10.826s)
        test/e2e/network/loadbalancer.go:1466

      Spec Goroutine
      goroutine 956 [select]
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.ExecWithFullOutput({0xc000d611e0?, 0x0?})
          test/e2e/framework/kubectl/builder.go:125
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.Exec(...)
          test/e2e/framework/kubectl/builder.go:107
        k8s.io/kubernetes/test/e2e/framework/kubectl.RunKubectl({0xc00450e7d0?, 0x1?}, {0xc004e91ad8?, 0x101010020?, 0x0?})
          test/e2e/framework/kubectl/builder.go:154
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmd(...)
          test/e2e/framework/pod/output/output.go:82
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6.3()
          test/e2e/network/loadbalancer.go:1468
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1({0x2742871, 0x0})
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:222
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext({0x7fe0bc8?, 0xc00012e000?}, 0xc00456f0c0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:235
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc00012e000}, 0xc004f8acc0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:662
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc00012e000}, 0xb0?, 0x2fd9d05?, 0x28?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc00012e000}, 0x0?, 0xc004e91d00?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc00141b080?, 0x78?, 0x0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6()
          test/e2e/network/loadbalancer.go:1467
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004506c00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:43:02.196: INFO: rc: 1
Nov 26 12:43:02.196: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:03.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:04.078: INFO: rc: 1
Nov 26 12:43:04.078: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:05.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:06.102: INFO: rc: 1
Nov 26 12:43:06.102: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:07.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:08.032: INFO: rc: 1
Nov 26 12:43:08.032: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:09.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:10.166: INFO: rc: 1
Nov 26 12:43:10.166: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:11.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:12.119: INFO: rc: 1
Nov 26 12:43:12.119: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:13.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:14.080: INFO: rc: 1
Nov 26 12:43:14.080: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:15.639: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:16.213: INFO: rc: 1
Nov 26 12:43:16.213: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:17.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:18.092: INFO: rc: 1
Nov 26 12:43:18.092: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:19.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:20.219: INFO: rc: 1
Nov 26 12:43:20.219: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:21.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
------------------------------
Progress Report for Ginkgo Process #6
Automatically polling progress:
  [sig-network] LoadBalancers ESIPP [Slow] should work from pods (Spec Runtime: 6m0.631s)
    test/e2e/network/loadbalancer.go:1422
    In [It] (Node Runtime: 6m0.009s)
      test/e2e/network/loadbalancer.go:1422
      At [By Step] Hitting external lb 35.247.92.206 from pod pause-pod-deployment-7c748f845d-rtr6l on node bootstrap-e2e-minion-group-tvmd (Step Runtime: 3m30.829s)
        test/e2e/network/loadbalancer.go:1466

      Spec Goroutine
      goroutine 956 [select]
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.ExecWithFullOutput({0xc000d61600?, 0x0?})
          test/e2e/framework/kubectl/builder.go:125
        k8s.io/kubernetes/test/e2e/framework/kubectl.KubectlBuilder.Exec(...)
          test/e2e/framework/kubectl/builder.go:107
        k8s.io/kubernetes/test/e2e/framework/kubectl.RunKubectl({0xc00450e7d0?, 0x1?}, {0xc004e91ad8?, 0x101010020?, 0x0?})
          test/e2e/framework/kubectl/builder.go:154
        k8s.io/kubernetes/test/e2e/framework/pod/output.RunHostCmd(...)
          test/e2e/framework/pod/output/output.go:82
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6.3()
          test/e2e/network/loadbalancer.go:1468
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1({0x2742871, 0x0})
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:222
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext({0x7fe0bc8?, 0xc00012e000?}, 0xc00456f0c0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:235
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x7fe0bc8, 0xc00012e000}, 0xc004f8acc0, 0x2fdb16a?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:662
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x7fe0bc8, 0xc00012e000}, 0xb0?, 0x2fd9d05?, 0x28?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x7fe0bc8, 0xc00012e000}, 0x0?, 0xc004e91d00?, 0x262a967?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:528
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc00141b080?, 0x78?, 0x0?)
          vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:514
      > k8s.io/kubernetes/test/e2e/network.glob..func20.6()
          test/e2e/network/loadbalancer.go:1467
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc004506c00})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 26 12:43:22.131: INFO: rc: 1
Nov 26 12:43:22.131: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:23.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:24.047: INFO: rc: 1
Nov 26 12:43:24.047: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip:
Command stdout:

stderr:
Error from server: error dialing backend: No agent available

error:
exit status 1, retry until timeout
Nov 26 12:43:25.638: INFO: Running '/workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/sh -x -c curl -q -s --connect-timeout 30 35.247.92.206:80/clientip'
Nov 26 12:43:26.157: INFO: rc: 1
Nov 26 12:43:26.157: INFO: got err: error running /workspace/github.com/containerd/containerd/kubernetes/platforms/linux/amd64/kubectl --server=https://34.145.15.180 --kubeconfig=/workspace/.kube/config --namespace=esipp-3856 exec pause-pod-deployment-7c748f845d-rtr6l -- /bin/

Result	FAILURE
Tests	198 failed / 139 succeeded
Started	2022-11-26 12:26
Elapsed	1h5m
Revision
Builder	7bdb70ce-6d85-11ed-a6da-22215f11743a
control_plane_node_os_image	cos-97-16919-103-16
infra-commit	b02ce7741
job-version	v1.27.0-alpha.0.50+70617042976dc1
kubetest-version	v20221116-7c85504268
repo	github.com/containerd/containerd
repo-commit	d06318622178a2209338d7f57eb8800b9f2e72fb
repos	{u'github.com/containerd/containerd': u'main'}
revision	v1.27.0-alpha.0.50+70617042976dc1
worker_node_os_image	cos-97-16919-103-16