ci-kubernetes-e2e-gci-gce #1637248464293203968
Recent runs || View in Spyglass
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sPort\sforwarding\sWith\sa\sserver\slistening\son\slocalhost\sshould\ssupport\sforwarding\sover\swebsockets$'
[FAILED] Failed to open websocket to wss://34.145.101.188/api/v1/namespaces/port-forwarding-2479/pods/pfpod/portforward?ports=80: websocket.Dial wss://34.145.101.188/api/v1/namespaces/port-forwarding-2479/pods/pfpod/portforward?ports=80: bad status In [It] at: test/e2e/kubectl/portforward.go:385 @ 03/19/23 00:31:08.617from junit_01.xml
> Enter [BeforeEach] [sig-cli] Kubectl Port forwarding - set up framework | framework.go:191 @ 03/19/23 00:30:48.236 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:48.236 Mar 19 00:30:48.236: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename port-forwarding - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:48.238 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:48.517 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:48.717 < Exit [BeforeEach] [sig-cli] Kubectl Port forwarding - set up framework | framework.go:191 @ 03/19/23 00:30:48.812 (576ms) > Enter [BeforeEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.812 < Exit [BeforeEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.812 (0s) > Enter [It] should support forwarding over websockets - test/e2e/kubectl/portforward.go:492 @ 03/19/23 00:30:48.812 Mar 19 00:30:48.812: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Creating the pod - test/e2e/kubectl/portforward.go:366 @ 03/19/23 00:30:48.813 [FAILED] Failed to open websocket to wss://34.145.101.188/api/v1/namespaces/port-forwarding-2479/pods/pfpod/portforward?ports=80: websocket.Dial wss://34.145.101.188/api/v1/namespaces/port-forwarding-2479/pods/pfpod/portforward?ports=80: bad status In [It] at: test/e2e/kubectl/portforward.go:385 @ 03/19/23 00:31:08.617 < Exit [It] should support forwarding over websockets - test/e2e/kubectl/portforward.go:492 @ 03/19/23 00:31:08.617 (19.805s) > Enter [AfterEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:08.617 Mar 19 00:31:08.617: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:08.861 (245ms) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:08.861 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:08.861 (0s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - dump namespaces | framework.go:209 @ 03/19/23 00:31:08.861 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:08.861 STEP: Collecting events from namespace "port-forwarding-2479". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:08.861 STEP: Found 8 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:09.03 Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:48 +0000 UTC - event for pfpod: {default-scheduler } Scheduled: Successfully assigned port-forwarding-2479/pfpod to bootstrap-e2e-minion-group-g5rl Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-g5rl} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-g5rl} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 6.055409728s (6.055437123s including waiting) Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-g5rl} Created: Created container readiness Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-g5rl} Started: Started container readiness Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-g5rl} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-g5rl} Created: Created container portforwardtester Mar 19 00:31:09.030: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-g5rl} Started: Started container portforwardtester Mar 19 00:31:09.136: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:09.136: INFO: pfpod bootstrap-e2e-minion-group-g5rl Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:48 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:31:01 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:31:01 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:48 +0000 UTC }] Mar 19 00:31:09.136: INFO: Mar 19 00:31:09.643: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:09.733: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:09.733: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:09.822: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:14.930: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:14.930: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:14.972: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:14.973: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:15.017: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:15.112: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:15.112: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:15.112: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:15.112: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:15.112: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-6gswg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-slljn started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-gfsns started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:15.112: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:15.112: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-5vxl5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-bgwp2 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-pnjjn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-wzssg started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-jdnxl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-gps48 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:15.112: INFO: hostexec-bootstrap-e2e-minion-group-494g-jd2vs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:15.112: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-rpvgh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-pd2gt started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-8vdgj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-wpfgd started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-cn9zr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-hd6xx started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-x5mkh started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-4dwp5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-nbwvl started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-vxwr9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-vbtkv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:15.112: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:15.112: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:15.112: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:15.112: INFO: simpletest.rc-7vn45 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:15.112: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.734: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:17.734: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:17.776: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 2324 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:11 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7 kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768 kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768,DevicePath:,},},Config:nil,},} Mar 19 00:31:17.776: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:17.823: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:17.921: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:17.921: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:17.921: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:17.921: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Init container init-volume-inlinevolume-km5v ready: true, restart count 0 Mar 19 00:31:17.921: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-69cfd started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-qnn4q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-w8jmk started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-frh8z started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: pod-subpath-test-dynamicpv-w4tj started at 2023-03-19 00:31:10 +0000 UTC (2+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Init container init-volume-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Init container test-init-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container test-container-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:17.921: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:17.921: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:17.921: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:17.921: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-6x66r started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: pod-55f6ff00-b7bd-449e-a96e-935dccef36ac started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:17.921: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-7sxn7 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-rsmrn started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:17.921: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:17.921: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:17.921: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:17.921: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:17.921: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-xdf8g started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-5jdgw started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: pod-7d5a3317-e722-4fa7-8cbf-8ed56da69143 started at 2023-03-19 00:31:11 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:17.921: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:17.921: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:17.921: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:17.921: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:17.921: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:17.921: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-b987l started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:17.921: INFO: simpletest.rc-4l96m started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:17.921: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.027: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:23.027: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:23.086: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:23.086: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:23.141: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:23.255: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:23.255: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:23.255: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:23.255: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:23.255: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:23.255: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:23.255: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container volume-tester ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: pod-submit-remove-f472c6a1-a8de-414e-9c1f-f9086d14466f started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container pause ready: false, restart count 0 Mar 19 00:31:23.255: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container c ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: httpd-deployment-55bcbdf6f4-nzqjk started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:23.255: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:23.255: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:23.255: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:23.255: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:23.255: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:23.255: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:23.255: INFO: var-expansion-96397faa-9dcb-440b-86ce-3f5b99f6d397 started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container dapi-container ready: false, restart count 0 Mar 19 00:31:23.255: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:23.255: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:23.255: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:23.255: INFO: pod-client started at 2023-03-19 00:31:13 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:23.255: INFO: Container pod-client ready: false, restart count 0 Mar 19 00:31:24.271: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:24.271: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:24.419: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:24.420: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:24.526: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:24.703: INFO: simpletest.rc-s7q9z started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.703: INFO: pod-de4e88ee-387a-42e0-a9e4-8a03876dc49f started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container write-pod ready: true, restart count 0 Mar 19 00:31:24.703: INFO: simpletest.rc-qz8x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.703: INFO: simpletest.rc-vpkhp started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:24.703: INFO: httpd-deployment-55bcbdf6f4-z75nw started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:24.703: INFO: metadata-proxy-v0.1-xn95s started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:24.703: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:24.703: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:24.703: INFO: e2e-net-client started at 2023-03-19 00:31:04 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container e2e-net-client ready: true, restart count 0 Mar 19 00:31:24.703: INFO: simpletest.rc-v2h4g started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.703: INFO: simpletest.rc-qt9cv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.703: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-k4vh7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:24.704: INFO: pvc-tester-n6v47 started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-928rb started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-r77f8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:24.704: INFO: external-provisioner-kh5pp started at 2023-03-19 00:30:46 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nfs-provisioner ready: true, restart count 0 Mar 19 00:31:24.704: INFO: e2e-net-exec started at 2023-03-19 00:30:58 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 19 00:31:24.704: INFO: kube-proxy-bootstrap-e2e-minion-group-zt76 started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-nb5tk started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-9kxh7 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-nrb9d started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-mlpmn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: konnectivity-agent-jl7mj started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:24.704: INFO: httpd-deployment-5cd84d4f9-pngsc started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-xdvlv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-vzz8p started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-r4rx7 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:24.704: INFO: implicit-nonroot-uid started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container implicit-nonroot-uid ready: false, restart count 0 Mar 19 00:31:24.704: INFO: hostexec-bootstrap-e2e-minion-group-zt76-6tdlz started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-pk55q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-zfk5k started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-sck2h started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-cm7k8 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:24.704: INFO: dns-test-cff8fe3e-c2c5-48b0-8dc1-48e6b5438fcc started at 2023-03-19 00:31:18 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:24.704: INFO: Container jessie-querier ready: false, restart count 0 Mar 19 00:31:24.704: INFO: Container querier ready: false, restart count 0 Mar 19 00:31:24.704: INFO: Container webserver ready: false, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-4m46r started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-j9xqq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-nlqsp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-lss94 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-bpdmm started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:24.704: INFO: simpletest.rc-m2wd9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:24.704: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:25.259: INFO: Latency metrics for node bootstrap-e2e-minion-group-zt76 END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:25.259 (16.398s) < Exit [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - dump namespaces | framework.go:209 @ 03/19/23 00:31:25.259 (16.398s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - tear down framework | framework.go:206 @ 03/19/23 00:31:25.259 STEP: Destroying namespace "port-forwarding-2479" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:25.259 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - tear down framework | framework.go:206 @ 03/19/23 00:31:25.412 (153ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.412 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.412 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sDNS\sshould\ssupport\sconfigurable\spod\sDNS\snameservers\s\[Conformance\]$'
[FAILED] failed to run command '/agnhost dns-suffix' on pod, stdout: , stderr: , err: error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout: error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout In [It] at: test/e2e/network/dns.go:437 @ 03/19/23 00:31:02.742from junit_01.xml
> Enter [BeforeEach] [sig-network] DNS - set up framework | framework.go:191 @ 03/19/23 00:30:46.433 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:46.433 Mar 19 00:30:46.433: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename dns - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:46.435 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:46.718 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:46.917 < Exit [BeforeEach] [sig-network] DNS - set up framework | framework.go:191 @ 03/19/23 00:30:47.117 (684ms) > Enter [BeforeEach] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:47.117 < Exit [BeforeEach] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:47.117 (0s) > Enter [It] should support configurable pod DNS nameservers [Conformance] - test/e2e/network/dns.go:407 @ 03/19/23 00:30:47.117 STEP: Creating a pod with dnsPolicy=None and customized dnsConfig... - test/e2e/network/dns.go:408 @ 03/19/23 00:30:47.117 Mar 19 00:30:47.217: INFO: Created pod &Pod{ObjectMeta:{test-dns-nameservers dns-8786 e1ef5cc1-38c6-4a10-a3e1-19a92ba1b220 947 0 2023-03-19 00:30:47 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update v1 2023-03-19 00:30:47 +0000 UTC FieldsV1 {"f:spec":{"f:containers":{"k:{\"name\":\"agnhost-container\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsConfig":{".":{},"f:nameservers":{},"f:searches":{}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}} }]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:kube-api-access-zk6vp,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:nil,NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:&ProjectedVolumeSource{Sources:[]VolumeProjection{VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:nil,ServiceAccountToken:&ServiceAccountTokenProjection{Audience:,ExpirationSeconds:*3607,Path:token,},},VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:&ConfigMapProjection{LocalObjectReference:LocalObjectReference{Name:kube-root-ca.crt,},Items:[]KeyToPath{KeyToPath{Key:ca.crt,Path:ca.crt,Mode:nil,},},Optional:nil,},ServiceAccountToken:nil,},VolumeProjection{Secret:nil,DownwardAPI:&DownwardAPIProjection{Items:[]DownwardAPIVolumeFile{DownwardAPIVolumeFile{Path:namespace,FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,Mode:nil,},},},ConfigMap:nil,ServiceAccountToken:nil,},},DefaultMode:*420,},StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:agnhost-container,Image:registry.k8s.io/e2e-test-images/agnhost:2.43,Command:[],Args:[pause],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-zk6vp,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:None,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:&PodDNSConfig{Nameservers:[1.1.1.1],Searches:[resolv.conf.local],Options:[]PodDNSConfigOption{},},ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,OS:nil,HostUsers:nil,SchedulingGates:[]PodSchedulingGate{},ResourceClaims:[]PodResourceClaim{},},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{},Message:,Reason:,HostIP:,PodIP:,StartTime:<nil>,ContainerStatuses:[]ContainerStatus{},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{},EphemeralContainerStatuses:[]ContainerStatus{},Resize:,},} STEP: Verifying customized DNS suffix list is configured on pod... - test/e2e/network/dns.go:441 @ 03/19/23 00:30:57.6 Mar 19 00:30:57.600: INFO: ExecWithOptions {Command:[/agnhost dns-suffix] Namespace:dns-8786 PodName:test-dns-nameservers ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 19 00:30:57.601: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:30:57.602: INFO: ExecWithOptions: Clientset creation Mar 19 00:30:57.602: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/dns-8786/pods/test-dns-nameservers/exec?command=%2Fagnhost&command=dns-suffix&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:02.741: INFO: Unexpected error: failed to run command '/agnhost dns-suffix' on pod, stdout: , stderr: , err: error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout: <*errors.StatusError | 0xc0004757c0>: error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] failed to run command '/agnhost dns-suffix' on pod, stdout: , stderr: , err: error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout: error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout In [It] at: test/e2e/network/dns.go:437 @ 03/19/23 00:31:02.742 < Exit [It] should support configurable pod DNS nameservers [Conformance] - test/e2e/network/dns.go:407 @ 03/19/23 00:31:02.742 (15.625s) > Enter [AfterEach] [sig-network] DNS - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:02.742 Mar 19 00:31:02.742: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-network] DNS - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:02.828 (86ms) > Enter [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:420 @ 03/19/23 00:31:02.828 Mar 19 00:31:02.828: INFO: Deleting pod test-dns-nameservers... < Exit [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:420 @ 03/19/23 00:31:02.889 (61ms) > Enter [DeferCleanup (Each)] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:02.889 < Exit [DeferCleanup (Each)] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:02.889 (0s) > Enter [DeferCleanup (Each)] [sig-network] DNS - dump namespaces | framework.go:209 @ 03/19/23 00:31:02.889 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:02.889 STEP: Collecting events from namespace "dns-8786". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:02.889 STEP: Found 6 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:02.934 Mar 19 00:31:02.934: INFO: At 2023-03-19 00:30:47 +0000 UTC - event for test-dns-nameservers: {default-scheduler } Scheduled: Successfully assigned dns-8786/test-dns-nameservers to bootstrap-e2e-minion-group-zt76 Mar 19 00:31:02.934: INFO: At 2023-03-19 00:30:48 +0000 UTC - event for test-dns-nameservers: {kubelet bootstrap-e2e-minion-group-zt76} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:02.934: INFO: At 2023-03-19 00:30:54 +0000 UTC - event for test-dns-nameservers: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 6.740861631s (6.740872274s including waiting) Mar 19 00:31:02.934: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for test-dns-nameservers: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container agnhost-container Mar 19 00:31:02.934: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for test-dns-nameservers: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container agnhost-container Mar 19 00:31:02.934: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for test-dns-nameservers: {kubelet bootstrap-e2e-minion-group-zt76} Killing: Stopping container agnhost-container Mar 19 00:31:02.976: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:02.976: INFO: Mar 19 00:31:03.024: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:03.066: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:03.066: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:03.109: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:08.207: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:08.207: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.280: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:08.280: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.388: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.632: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:08.632: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:08.632: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:08.632: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:08.632: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-6gswg started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.632: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:08.632: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:08.632: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-5vxl5 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.632: INFO: hostexec-bootstrap-e2e-minion-group-494g-jd2vs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container agnhost-container ready: false, restart count 0 Mar 19 00:31:08.632: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-rpvgh started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: simpletest.rc-wzssg started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: simpletest.rc-pd2gt started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: simpletest.rc-8vdgj started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: simpletest.rc-wpfgd started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-nbwvl started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.632: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-4dwp5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.632: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:08.632: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:08.632: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:08.632: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.632: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:08.632: INFO: simpletest.rc-7vn45 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:11.155: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:11.156: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:11.199: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 1511 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:11.199: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:11.248: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:11.393: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:11.393: INFO: simpletest.rc-7sxn7 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.393: INFO: simpletest.rc-rsmrn started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.393: INFO: simpletest.rc-xdf8g started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.393: INFO: simpletest.rc-5jdgw started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.393: INFO: pod-7d5a3317-e722-4fa7-8cbf-8ed56da69143 started at 2023-03-19 00:31:11 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:11.393: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:11.393: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:11.393: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:11.393: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:11.393: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:11.393: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:11.393: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.393: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.393: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:11.394: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:11.394: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:11.394: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:11.394: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-b987l started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-4l96m started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:11.394: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-69cfd started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:11.394: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:11.394: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:11.394: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Init container init-volume-inlinevolume-km5v ready: true, restart count 0 Mar 19 00:31:11.394: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-qnn4q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-w8jmk started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-frh8z started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: pod-subpath-test-dynamicpv-w4tj started at 2023-03-19 00:31:10 +0000 UTC (2+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Init container init-volume-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Init container test-init-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Container test-container-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:11.394: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:11.394: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:11.394: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:11.394: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:11.394: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:11.394: INFO: simpletest.rc-6x66r started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.394: INFO: pod-55f6ff00-b7bd-449e-a96e-935dccef36ac started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.394: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:18.762: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:18.762: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:18.838: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:18.839: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:18.921: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:19.114: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.114: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:19.114: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:19.114: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:19.114: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:19.114: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:19.114: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container c ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: httpd-deployment-55bcbdf6f4-nzqjk started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:19.114: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:19.114: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:19.114: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.114: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:19.114: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.114: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:19.114: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:19.114: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.114: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:19.114: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.114: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.115: INFO: pod-client started at 2023-03-19 00:31:13 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.115: INFO: Container pod-client ready: false, restart count 0 Mar 19 00:31:19.115: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.115: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.115: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.115: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.115: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.115: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.115: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.115: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.413: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:20.413: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.520: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:20.520: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.581: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.753: INFO: simpletest.rc-cm7k8 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: dns-test-cff8fe3e-c2c5-48b0-8dc1-48e6b5438fcc started at 2023-03-19 00:31:18 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:20.753: INFO: Container jessie-querier ready: false, restart count 0 Mar 19 00:31:20.753: INFO: Container querier ready: false, restart count 0 Mar 19 00:31:20.753: INFO: Container webserver ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-4m46r started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-j9xqq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-nlqsp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-lss94 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-bpdmm started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-m2wd9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-s7q9z started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: pod-de4e88ee-387a-42e0-a9e4-8a03876dc49f started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container write-pod ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-qz8x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-vpkhp started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: httpd-deployment-55bcbdf6f4-z75nw started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:20.753: INFO: metadata-proxy-v0.1-xn95s started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:20.753: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:20.753: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:20.753: INFO: e2e-net-client started at 2023-03-19 00:31:04 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container e2e-net-client ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-v2h4g started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-qt9cv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-k4vh7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: pvc-tester-n6v47 started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-928rb started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-r77f8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: external-provisioner-kh5pp started at 2023-03-19 00:30:46 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nfs-provisioner ready: true, restart count 0 Mar 19 00:31:20.753: INFO: e2e-net-exec started at 2023-03-19 00:30:58 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 19 00:31:20.753: INFO: kube-proxy-bootstrap-e2e-minion-group-zt76 started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-nb5tk started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-9kxh7 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-nrb9d started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-mlpmn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: konnectivity-agent-jl7mj started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:20.753: INFO: httpd-deployment-5cd84d4f9-pngsc started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-xdvlv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-vzz8p started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-r4rx7 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: implicit-nonroot-uid started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container implicit-nonroot-uid ready: false, restart count 0 Mar 19 00:31:20.753: INFO: hostexec-bootstrap-e2e-minion-group-zt76-6tdlz started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-pk55q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-zfk5k started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.753: INFO: simpletest.rc-sck2h started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.753: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:25.119: INFO: Latency metrics for node bootstrap-e2e-minion-group-zt76 END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:25.119 (22.23s) < Exit [DeferCleanup (Each)] [sig-network] DNS - dump namespaces | framework.go:209 @ 03/19/23 00:31:25.119 (22.23s) > Enter [DeferCleanup (Each)] [sig-network] DNS - tear down framework | framework.go:206 @ 03/19/23 00:31:25.119 STEP: Destroying namespace "dns-8786" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:25.119 < Exit [DeferCleanup (Each)] [sig-network] DNS - tear down framework | framework.go:206 @ 03/19/23 00:31:25.221 (102ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.221 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.221 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sDNS\sshould\ssupport\sconfigurable\spod\sresolv\.conf$'
[FAILED] failed to examine resolv,conf file on pod, stdout: , stderr: , err: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout In [It] at: test/e2e/network/dns.go:524 @ 03/19/23 00:31:06.746from junit_01.xml
> Enter [BeforeEach] [sig-network] DNS - set up framework | framework.go:191 @ 03/19/23 00:30:48.421 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:48.421 Mar 19 00:30:48.421: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename dns - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:48.422 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:48.718 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:48.828 < Exit [BeforeEach] [sig-network] DNS - set up framework | framework.go:191 @ 03/19/23 00:30:48.929 (509ms) > Enter [BeforeEach] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.929 < Exit [BeforeEach] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.93 (0s) > Enter [It] should support configurable pod resolv.conf - test/e2e/network/dns.go:454 @ 03/19/23 00:30:48.93 STEP: Preparing a test DNS service with injected DNS names... - test/e2e/network/dns.go:455 @ 03/19/23 00:30:48.93 Mar 19 00:30:49.073: INFO: Created pod &Pod{ObjectMeta:{e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda dns-1378 52687dfe-bc6c-4731-af89-4ae252bb0865 1081 0 2023-03-19 00:30:49 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update v1 2023-03-19 00:30:49 +0000 UTC FieldsV1 {"f:spec":{"f:containers":{"k:{\"name\":\"agnhost-container\"}":{".":{},"f:args":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/etc/coredns\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"coredns-config\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:name":{}},"f:name":{}}}}} }]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:coredns-config,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:nil,NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:&ConfigMapVolumeSource{LocalObjectReference:LocalObjectReference{Name:e2e-coredns-configmap-2sn6d,},Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,Ephemeral:nil,},},Volume{Name:kube-api-access-5j6wj,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:nil,NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:&ProjectedVolumeSource{Sources:[]VolumeProjection{VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:nil,ServiceAccountToken:&ServiceAccountTokenProjection{Audience:,ExpirationSeconds:*3607,Path:token,},},VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:&ConfigMapProjection{LocalObjectReference:LocalObjectReference{Name:kube-root-ca.crt,},Items:[]KeyToPath{KeyToPath{Key:ca.crt,Path:ca.crt,Mode:nil,},},Optional:nil,},ServiceAccountToken:nil,},VolumeProjection{Secret:nil,DownwardAPI:&DownwardAPIProjection{Items:[]DownwardAPIVolumeFile{DownwardAPIVolumeFile{Path:namespace,FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,Mode:nil,},},},ConfigMap:nil,ServiceAccountToken:nil,},},DefaultMode:*420,},StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:agnhost-container,Image:registry.k8s.io/e2e-test-images/agnhost:2.43,Command:[/coredns],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:coredns-config,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:kube-api-access-5j6wj,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:Default,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,OS:nil,HostUsers:nil,SchedulingGates:[]PodSchedulingGate{},ResourceClaims:[]PodResourceClaim{},},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{},Message:,Reason:,HostIP:,PodIP:,StartTime:<nil>,ContainerStatuses:[]ContainerStatus{},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{},EphemeralContainerStatuses:[]ContainerStatus{},Resize:,},} Mar 19 00:30:57.410: INFO: testServerIP is 10.64.0.5 STEP: Creating a pod with dnsPolicy=None and customized dnsConfig... - test/e2e/network/dns.go:489 @ 03/19/23 00:30:57.41 Mar 19 00:30:57.455: INFO: Created pod &Pod{ObjectMeta:{e2e-dns-utils dns-1378 413071bf-ecfc-4154-b675-32a92dde6a4b 1454 0 2023-03-19 00:30:57 +0000 UTC <nil> <nil> map[] map[] [] [] [{e2e.test Update v1 2023-03-19 00:30:57 +0000 UTC FieldsV1 {"f:spec":{"f:containers":{"k:{\"name\":\"agnhost-container\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:securityContext":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsConfig":{".":{},"f:nameservers":{},"f:options":{},"f:searches":{}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}} }]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:kube-api-access-5p58q,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:nil,NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:&ProjectedVolumeSource{Sources:[]VolumeProjection{VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:nil,ServiceAccountToken:&ServiceAccountTokenProjection{Audience:,ExpirationSeconds:*3607,Path:token,},},VolumeProjection{Secret:nil,DownwardAPI:nil,ConfigMap:&ConfigMapProjection{LocalObjectReference:LocalObjectReference{Name:kube-root-ca.crt,},Items:[]KeyToPath{KeyToPath{Key:ca.crt,Path:ca.crt,Mode:nil,},},Optional:nil,},ServiceAccountToken:nil,},VolumeProjection{Secret:nil,DownwardAPI:&DownwardAPIProjection{Items:[]DownwardAPIVolumeFile{DownwardAPIVolumeFile{Path:namespace,FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,Mode:nil,},},},ConfigMap:nil,ServiceAccountToken:nil,},},DefaultMode:*420,},StorageOS:nil,CSI:nil,Ephemeral:nil,},},},Containers:[]Container{Container{Name:agnhost-container,Image:registry.k8s.io/e2e-test-images/agnhost:2.43,Command:[],Args:[pause],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-5p58q,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},},},RestartPolicy:Always,TerminationGracePeriodSeconds:*0,ActiveDeadlineSeconds:nil,DNSPolicy:None,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:&PodDNSConfig{Nameservers:[10.64.0.5],Searches:[resolv.conf.local],Options:[]PodDNSConfigOption{PodDNSConfigOption{Name:ndots,Value:*2,},},},ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},SetHostnameAsFQDN:nil,OS:nil,HostUsers:nil,SchedulingGates:[]PodSchedulingGate{},ResourceClaims:[]PodResourceClaim{},},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{},Message:,Reason:,HostIP:,PodIP:,StartTime:<nil>,ContainerStatuses:[]ContainerStatus{},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{},EphemeralContainerStatuses:[]ContainerStatus{},Resize:,},} STEP: Verifying customized DNS option is configured on pod... - test/e2e/network/dns.go:513 @ 03/19/23 00:31:01.594 Mar 19 00:31:01.594: INFO: ExecWithOptions {Command:[cat /etc/resolv.conf] Namespace:dns-1378 PodName:e2e-dns-utils ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 19 00:31:01.594: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:31:01.595: INFO: ExecWithOptions: Clientset creation Mar 19 00:31:01.595: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/dns-1378/pods/e2e-dns-utils/exec?command=cat&command=%2Fetc%2Fresolv.conf&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:06.746: INFO: Unexpected error: failed to examine resolv,conf file on pod, stdout: , stderr: , err: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout: <*errors.StatusError | 0xc00026f5e0>: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] failed to examine resolv,conf file on pod, stdout: , stderr: , err: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout In [It] at: test/e2e/network/dns.go:524 @ 03/19/23 00:31:06.746 < Exit [It] should support configurable pod resolv.conf - test/e2e/network/dns.go:454 @ 03/19/23 00:31:06.746 (17.817s) > Enter [AfterEach] [sig-network] DNS - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:06.746 Mar 19 00:31:06.746: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-network] DNS - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:06.83 (84ms) > Enter [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:506 @ 03/19/23 00:31:06.83 Mar 19 00:31:06.830: INFO: Deleting pod e2e-dns-utils... < Exit [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:506 @ 03/19/23 00:31:06.927 (97ms) > Enter [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:476 @ 03/19/23 00:31:06.927 Mar 19 00:31:06.927: INFO: Deleting pod e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda... < Exit [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:476 @ 03/19/23 00:31:06.995 (69ms) > Enter [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:467 @ 03/19/23 00:31:06.995 Mar 19 00:31:06.996: INFO: Deleting configmap e2e-coredns-configmap-2sn6d... < Exit [DeferCleanup (Each)] [sig-network] DNS - test/e2e/network/dns.go:467 @ 03/19/23 00:31:07.115 (120ms) > Enter [DeferCleanup (Each)] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:07.115 < Exit [DeferCleanup (Each)] [sig-network] DNS - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:07.115 (0s) > Enter [DeferCleanup (Each)] [sig-network] DNS - dump namespaces | framework.go:209 @ 03/19/23 00:31:07.115 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:07.115 STEP: Collecting events from namespace "dns-1378". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:07.115 STEP: Found 11 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:07.337 Mar 19 00:31:07.337: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda: {default-scheduler } Scheduled: Successfully assigned dns-1378/e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda to bootstrap-e2e-minion-group-zt76 Mar 19 00:31:07.337: INFO: At 2023-03-19 00:30:50 +0000 UTC - event for e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda: {kubelet bootstrap-e2e-minion-group-zt76} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:07.337: INFO: At 2023-03-19 00:30:54 +0000 UTC - event for e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 4.415215868s (4.415224698s including waiting) Mar 19 00:31:07.337: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container agnhost-container Mar 19 00:31:07.337: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container agnhost-container Mar 19 00:31:07.337: INFO: At 2023-03-19 00:30:57 +0000 UTC - event for e2e-dns-utils: {default-scheduler } Scheduled: Successfully assigned dns-1378/e2e-dns-utils to bootstrap-e2e-minion-group-494g Mar 19 00:31:07.337: INFO: At 2023-03-19 00:30:59 +0000 UTC - event for e2e-dns-utils: {kubelet bootstrap-e2e-minion-group-494g} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 19 00:31:07.337: INFO: At 2023-03-19 00:31:00 +0000 UTC - event for e2e-dns-utils: {kubelet bootstrap-e2e-minion-group-494g} Created: Created container agnhost-container Mar 19 00:31:07.337: INFO: At 2023-03-19 00:31:01 +0000 UTC - event for e2e-dns-utils: {kubelet bootstrap-e2e-minion-group-494g} Started: Started container agnhost-container Mar 19 00:31:07.337: INFO: At 2023-03-19 00:31:06 +0000 UTC - event for e2e-configmap-dns-server-a0145ef4-bf7e-4f16-9cf6-a3371289feda: {kubelet bootstrap-e2e-minion-group-zt76} Killing: Stopping container agnhost-container Mar 19 00:31:07.337: INFO: At 2023-03-19 00:31:06 +0000 UTC - event for e2e-dns-utils: {kubelet bootstrap-e2e-minion-group-494g} Killing: Stopping container agnhost-container Mar 19 00:31:07.493: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:07.493: INFO: Mar 19 00:31:07.666: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:07.828: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:07.828: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:08.066: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:13.226: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:13.226: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:13.278: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:13.278: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:13.325: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:13.428: INFO: simpletest.rc-vxwr9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-4dwp5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-nbwvl started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-7vn45 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-vbtkv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:13.428: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:13.428: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-slljn started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-gfsns started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:13.428: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:13.428: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:13.428: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:13.428: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-6gswg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:13.428: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:13.428: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-5vxl5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-bgwp2 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-pnjjn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-rpvgh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-wzssg started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-jdnxl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-gps48 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:13.428: INFO: hostexec-bootstrap-e2e-minion-group-494g-jd2vs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:13.428: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-cn9zr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-pd2gt started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-8vdgj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-wpfgd started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-x5mkh started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:13.428: INFO: simpletest.rc-hd6xx started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:13.428: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.010: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:14.010: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:14.056: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 2098 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:11 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768,DevicePath:,},},Config:nil,},} Mar 19 00:31:14.056: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:14.101: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:14.193: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:14.193: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:14.193: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:14.193: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-b987l started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-4l96m started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:14.193: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-69cfd started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:14.193: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:14.193: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:14.193: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Init container init-volume-inlinevolume-km5v ready: true, restart count 0 Mar 19 00:31:14.193: INFO: Container test-container-subpath-inlinevolume-km5v ready: true, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-qnn4q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-w8jmk started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-frh8z started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: pod-subpath-test-dynamicpv-w4tj started at 2023-03-19 00:31:10 +0000 UTC (2+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Init container init-volume-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Init container test-init-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container test-container-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:14.193: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-6x66r started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: pod-55f6ff00-b7bd-449e-a96e-935dccef36ac started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:14.193: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:14.193: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-rsmrn started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-7sxn7 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:14.193: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-xdf8g started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: simpletest.rc-5jdgw started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:14.193: INFO: pod-7d5a3317-e722-4fa7-8cbf-8ed56da69143 started at 2023-03-19 00:31:11 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:14.193: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:14.193: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:14.193: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:14.193: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:14.193: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:18.770: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:18.770: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:18.839: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:18.840: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:18.908: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:19.008: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:19.008: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:19.008: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:19.008: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:19.008: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.008: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:19.008: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:19.008: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.008: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: pod-client started at 2023-03-19 00:31:13 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container pod-client ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.008: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:19.008: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:19.008: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:19.008: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:19.008: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:19.008: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container c ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.008: INFO: httpd-deployment-55bcbdf6f4-nzqjk started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.008: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:20.418: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:20.418: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.561: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:20.561: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.604: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.761: INFO: simpletest.rc-m2wd9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-cm7k8 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: dns-test-cff8fe3e-c2c5-48b0-8dc1-48e6b5438fcc started at 2023-03-19 00:31:18 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:20.761: INFO: Container jessie-querier ready: false, restart count 0 Mar 19 00:31:20.761: INFO: Container querier ready: false, restart count 0 Mar 19 00:31:20.761: INFO: Container webserver ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-4m46r started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-j9xqq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-nlqsp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-lss94 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-bpdmm started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-s7q9z started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: pod-de4e88ee-387a-42e0-a9e4-8a03876dc49f started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container write-pod ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-qz8x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-vpkhp started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: httpd-deployment-55bcbdf6f4-z75nw started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:20.761: INFO: metadata-proxy-v0.1-xn95s started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:20.761: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:20.761: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:20.761: INFO: e2e-net-client started at 2023-03-19 00:31:04 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container e2e-net-client ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-v2h4g started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-qt9cv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-k4vh7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: pvc-tester-n6v47 started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-928rb started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-r77f8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: external-provisioner-kh5pp started at 2023-03-19 00:30:46 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nfs-provisioner ready: true, restart count 0 Mar 19 00:31:20.761: INFO: e2e-net-exec started at 2023-03-19 00:30:58 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 19 00:31:20.761: INFO: kube-proxy-bootstrap-e2e-minion-group-zt76 started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-nb5tk started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-9kxh7 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-nrb9d started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-mlpmn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: implicit-nonroot-uid started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container implicit-nonroot-uid ready: false, restart count 0 Mar 19 00:31:20.761: INFO: konnectivity-agent-jl7mj started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:20.761: INFO: httpd-deployment-5cd84d4f9-pngsc started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-xdvlv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-vzz8p started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-r4rx7 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: hostexec-bootstrap-e2e-minion-group-zt76-6tdlz started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-pk55q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-zfk5k started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.761: INFO: simpletest.rc-sck2h started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.761: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:25.081: INFO: Latency metrics for node bootstrap-e2e-minion-group-zt76 END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:25.081 (17.966s) < Exit [DeferCleanup (Each)] [sig-network] DNS - dump namespaces | framework.go:209 @ 03/19/23 00:31:25.081 (17.966s) > Enter [DeferCleanup (Each)] [sig-network] DNS - tear down framework | framework.go:206 @ 03/19/23 00:31:25.081 STEP: Destroying namespace "dns-1378" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:25.081 < Exit [DeferCleanup (Each)] [sig-network] DNS - tear down framework | framework.go:206 @ 03/19/23 00:31:25.162 (80ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.162 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.162 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sEmptyDir\svolumes\sshould\ssupport\s\(root\,0666\,tmpfs\)\s\[LinuxOnly\]\s\[NodeConformance\]\s\[Conformance\]$'
[FAILED] failed to get logs from pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755 for test-container: an error on the server ("unknown") has prevented the request from succeeding (get pods pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755)
In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/19/23 00:31:07.655
> Enter [BeforeEach] [sig-storage] EmptyDir volumes - set up framework | framework.go:191 @ 03/19/23 00:30:49.06
STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:49.06
Mar 19 00:30:49.060: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename emptydir - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:49.061
STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:49.361
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:49.465
< Exit [BeforeEach] [sig-storage] EmptyDir volumes - set up framework | framework.go:191 @ 03/19/23 00:30:49.549 (489ms)
> Enter [BeforeEach] [sig-storage] EmptyDir volumes - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:49.549
< Exit [BeforeEach] [sig-storage] EmptyDir volumes - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:49.549 (0s)
> Enter [It] should support (root,0666,tmpfs) [LinuxOnly] [NodeConformance] [Conformance] - test/e2e/common/storage/empty_dir.go:107 @ 03/19/23 00:30:49.549
STEP: Creating a pod to test emptydir 0666 on tmpfs - test/e2e/framework/pod/output/output.go:243 @ 03/19/23 00:30:49.549
STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/19/23 00:31:01.964
Mar 19 00:31:02.008: INFO: Trying to get logs from node bootstrap-e2e-minion-group-zt76 pod pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755 container test-container: <nil>
Mar 19 00:31:07.116: INFO: Failed to get logs from node "bootstrap-e2e-minion-group-zt76" pod "pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755" container "test-container". an error on the server ("unknown") has prevented the request from succeeding (get pods pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755)
STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/19/23 00:31:07.116
Mar 19 00:31:07.654: INFO: Unexpected error:
<*fmt.wrapError | 0xc003282940>:
failed to get logs from pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755 for test-container: an error on the server ("unknown") has prevented the request from succeeding (get pods pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755)
{
msg: "failed to get logs from pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755 for test-container: an error on the server (\"unknown\") has prevented the request from succeeding (get pods pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755)",
err: <*errors.StatusError | 0xc000e392c0>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "an error on the server (\"unknown\") has prevented the request from succeeding (get pods pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755)",
Reason: "InternalError",
Details: {
Name: "pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755",
Group: "",
Kind: "pods",
UID: "",
Causes: [
{
Type: "UnexpectedServerResponse",
Message: "unknown",
Field: "",
},
],
RetryAfterSeconds: 0,
},
Code: 500,
},
},
}
[FAILED] failed to get logs from pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755 for test-container: an error on the server ("unknown") has prevented the request from succeeding (get pods pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755)
In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/19/23 00:31:07.655
< Exit [It] should support (root,0666,tmpfs) [LinuxOnly] [NodeConformance] [Conformance] - test/e2e/common/storage/empty_dir.go:107 @ 03/19/23 00:31:07.655 (18.106s)
> Enter [AfterEach] [sig-storage] EmptyDir volumes - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:07.655
Mar 19 00:31:07.655: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
< Exit [AfterEach] [sig-storage] EmptyDir volumes - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:07.828 (174ms)
> Enter [DeferCleanup (Each)] [sig-storage] EmptyDir volumes - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:07.828
< Exit [DeferCleanup (Each)] [sig-storage] EmptyDir volumes - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:07.828 (0s)
> Enter [DeferCleanup (Each)] [sig-storage] EmptyDir volumes - dump namespaces | framework.go:209 @ 03/19/23 00:31:07.828
STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:07.828
STEP: Collecting events from namespace "emptydir-4564". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:07.829
STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:07.978
Mar 19 00:31:07.978: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755: {default-scheduler } Scheduled: Successfully assigned emptydir-4564/pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755 to bootstrap-e2e-minion-group-zt76
Mar 19 00:31:07.978: INFO: At 2023-03-19 00:30:51 +0000 UTC - event for pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755: {kubelet bootstrap-e2e-minion-group-zt76} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43"
Mar 19 00:31:07.978: INFO: At 2023-03-19 00:30:54 +0000 UTC - event for pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 3.486833812s (3.486847029s including waiting)
Mar 19 00:31:07.978: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container test-container
Mar 19 00:31:07.978: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for pod-cdc47ca7-b9c7-4b02-b788-d2a5e810b755: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container test-container
Mar 19 00:31:08.097: INFO: POD NODE PHASE GRACE CONDITIONS
Mar 19 00:31:08.097: INFO:
Mar 19 00:31:08.204: INFO:
Logging node info for node bootstrap-e2e-master
Mar 19 00:31:08.276: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Mar 19 00:31:08.276: INFO:
Logging kubelet events for node bootstrap-e2e-master
Mar 19 00:31:08.407: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Mar 19 00:31:13.584: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout
Mar 19 00:31:13.584: INFO:
Logging node info for node bootstrap-e2e-minion-group-494g
Mar 19 00:31:13.627: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Mar 19 00:31:13.628: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-494g
Mar 19 00:31:13.673: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g
Mar 19 00:31:13.786: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-hd6xx started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-x5mkh started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-4dwp5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-nbwvl started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-vxwr9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-vbtkv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:13.786: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container coredns ready: true, restart count 0
Mar 19 00:31:13.786: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nfs-server ready: true, restart count 0
Mar 19 00:31:13.786: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container httpd ready: true, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-7vn45 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:13.786: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:13.786: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0
Mar 19 00:31:13.786: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-6gswg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-slljn started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-gfsns started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:13.786: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0
Mar 19 00:31:13.786: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container e2e-net-server ready: true, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-5vxl5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-bgwp2 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-pnjjn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-wzssg started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-jdnxl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-gps48 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:13.786: INFO: hostexec-bootstrap-e2e-minion-group-494g-jd2vs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:13.786: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container agnhost ready: true, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-rpvgh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-pd2gt started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-8vdgj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-wpfgd started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:13.786: INFO: simpletest.rc-cn9zr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:13.786: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.731: INFO:
Latency metrics for node bootstrap-e2e-minion-group-494g
Mar 19 00:31:17.731: INFO:
Logging node info for node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:17.772: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 2324 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:11 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7 kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768 kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768,DevicePath:,},},Config:nil,},}
Mar 19 00:31:17.772: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:17.818: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:17.920: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container pod-csi-inline-volumes ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-7sxn7 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-rsmrn started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-xdf8g started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-5jdgw started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: pod-7d5a3317-e722-4fa7-8cbf-8ed56da69143 started at 2023-03-19 00:31:11 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container write-pod ready: false, restart count 0
Mar 19 00:31:17.920: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:17.920: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container csi-attacher ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-provisioner ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-resizer ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-snapshotter ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container hostpath ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container liveness-probe ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container node-driver-registrar ready: false, restart count 0
Mar 19 00:31:17.920: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container csi-attacher ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-provisioner ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-resizer ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-snapshotter ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container hostpath ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container liveness-probe ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container node-driver-registrar ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:17.920: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:17.920: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nfs-server ready: true, restart count 0
Mar 19 00:31:17.920: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-b987l started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-4l96m started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container volume-snapshot-controller ready: true, restart count 0
Mar 19 00:31:17.920: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container default-http-backend ready: true, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-69cfd started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container portforwardtester ready: true, restart count 0
Mar 19 00:31:17.920: INFO: Container readiness ready: true, restart count 0
Mar 19 00:31:17.920: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Init container init-volume-inlinevolume-km5v ready: true, restart count 0
Mar 19 00:31:17.920: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container autoscaler ready: true, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-qnn4q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-w8jmk started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-frh8z started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: pod-subpath-test-dynamicpv-w4tj started at 2023-03-19 00:31:10 +0000 UTC (2+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Init container init-volume-dynamicpv-w4tj ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Init container test-init-subpath-dynamicpv-w4tj ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container test-container-subpath-dynamicpv-w4tj ready: false, restart count 0
Mar 19 00:31:17.920: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container coredns ready: true, restart count 0
Mar 19 00:31:17.920: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:17.920: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container csi-attacher ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-provisioner ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-resizer ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container csi-snapshotter ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container hostpath ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container liveness-probe ready: false, restart count 0
Mar 19 00:31:17.920: INFO: Container node-driver-registrar ready: false, restart count 0
Mar 19 00:31:17.920: INFO: simpletest.rc-6x66r started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:17.920: INFO: pod-55f6ff00-b7bd-449e-a96e-935dccef36ac started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:17.920: INFO: Container write-pod ready: false, restart count 0
Mar 19 00:31:19.483: INFO:
Latency metrics for node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:19.483: INFO:
Logging node info for node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:19.603: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},}
Mar 19 00:31:19.604: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:19.661: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:19.761: INFO: httpd-deployment-55bcbdf6f4-nzqjk started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container httpd ready: false, restart count 0
Mar 19 00:31:19.761: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container c ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container csi-provisioner ready: true, restart count 0
Mar 19 00:31:19.761: INFO: Container driver-registrar ready: true, restart count 0
Mar 19 00:31:19.761: INFO: Container mock ready: true, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:19.761: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container write-pod ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:19.761: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container metrics-server ready: true, restart count 0
Mar 19 00:31:19.761: INFO: Container metrics-server-nanny ready: true, restart count 0
Mar 19 00:31:19.761: INFO: var-expansion-96397faa-9dcb-440b-86ce-3f5b99f6d397 started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container dapi-container ready: false, restart count 0
Mar 19 00:31:19.761: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container csi-attacher ready: true, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: pod-client started at 2023-03-19 00:31:13 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container pod-client ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container volume-tester ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:19.761: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:19.761: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:19.761: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:19.761: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:19.761: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.761: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.094: INFO:
Latency metrics for node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:21.094: INFO:
Logging node info for node bootstrap-e2e-minion-group-zt76
Mar 19 00:31:21.257: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Mar 19 00:31:21.257: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-zt76
Mar 19 00:31:21.394: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76
Mar 19 00:31:21.676: INFO: simpletest.rc-4m46r started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-j9xqq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-nlqsp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-lss94 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-bpdmm started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-m2wd9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-cm7k8 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: dns-test-cff8fe3e-c2c5-48b0-8dc1-48e6b5438fcc started at 2023-03-19 00:31:18 +0000 UTC (0+3 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container jessie-querier ready: false, restart count 0
Mar 19 00:31:21.676: INFO: Container querier ready: false, restart count 0
Mar 19 00:31:21.676: INFO: Container webserver ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-s7q9z started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: pod-de4e88ee-387a-42e0-a9e4-8a03876dc49f started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container write-pod ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-qz8x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-vpkhp started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: httpd-deployment-55bcbdf6f4-z75nw started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container httpd ready: false, restart count 0
Mar 19 00:31:21.676: INFO: metadata-proxy-v0.1-xn95s started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:21.676: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:21.676: INFO: e2e-net-client started at 2023-03-19 00:31:04 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container e2e-net-client ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-v2h4g started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-qt9cv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-k4vh7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: pvc-tester-n6v47 started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container write-pod ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-928rb started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-r77f8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: external-provisioner-kh5pp started at 2023-03-19 00:30:46 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nfs-provisioner ready: true, restart count 0
Mar 19 00:31:21.676: INFO: e2e-net-exec started at 2023-03-19 00:30:58 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container e2e-net-exec ready: true, restart count 0
Mar 19 00:31:21.676: INFO: kube-proxy-bootstrap-e2e-minion-group-zt76 started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-nb5tk started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-9kxh7 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-nrb9d started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-mlpmn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: konnectivity-agent-jl7mj started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:21.676: INFO: httpd-deployment-5cd84d4f9-pngsc started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container httpd ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-xdvlv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-vzz8p started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-r4rx7 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: implicit-nonroot-uid started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container implicit-nonroot-uid ready: false, restart count 0
Mar 19 00:31:21.676: INFO: hostexec-bootstrap-e2e-minion-group-zt76-6tdlz started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-pk55q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-zfk5k started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:21.676: INFO: simpletest.rc-sck2h started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:21.676: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:25.095: INFO:
Latency metrics for node bootstrap-e2e-minion-group-zt76
END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:25.095 (17.266s)
< Exit [DeferCleanup (Each)] [sig-storage] EmptyDir volumes - dump namespaces | framework.go:209 @ 03/19/23 00:31:25.095 (17.266s)
> Enter [DeferCleanup (Each)] [sig-storage] EmptyDir volumes - tear down framework | framework.go:206 @ 03/19/23 00:31:25.095
STEP: Destroying namespace "emptydir-4564" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:25.095
< Exit [DeferCleanup (Each)] [sig-storage] EmptyDir volumes - tear down framework | framework.go:206 @ 03/19/23 00:31:25.204 (109ms)
> Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.205
< Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.205 (0s)
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sIn\-tree\sVolumes\s\[Driver\:\slocal\]\[LocalVolumeType\:\sblock\]\s\[Testpattern\:\sPre\-provisioned\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sexisting\sdirectories\swhen\sreadOnly\sspecified\sin\sthe\svolumeSource$'
[FAILED] error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:135 @ 03/19/23 00:31:02.548from junit_01.xml
> Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:50.786 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:50.786 (0s) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:50.786 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:50.786 Mar 19 00:30:50.786: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:50.787 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:50.912 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:50.993 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:51.078 (292ms) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:51.078 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:51.078 (0s) > Enter [It] should support existing directories when readOnly specified in the volumeSource - test/e2e/storage/testsuites/subpath.go:397 @ 03/19/23 00:30:51.078 Mar 19 00:30:51.121: INFO: In-tree plugin kubernetes.io/local-volume is not migrated, not validating any metrics STEP: Creating block device on node "bootstrap-e2e-minion-group-zt76" using path "/tmp/local-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e" - test/e2e/storage/utils/local.go:125 @ 03/19/23 00:30:51.121 Mar 19 00:30:57.365: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e && dd if=/dev/zero of=/tmp/local-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e/file bs=4096 count=5120 && losetup -f /tmp/local-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e/file] Namespace:provisioning-5171 PodName:hostexec-bootstrap-e2e-minion-group-zt76-vms47 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 19 00:30:57.365: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:30:57.366: INFO: ExecWithOptions: Clientset creation Mar 19 00:30:57.366: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/provisioning-5171/pods/hostexec-bootstrap-e2e-minion-group-zt76-vms47/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+-p+%2Ftmp%2Flocal-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e+%26%26+dd+if%3D%2Fdev%2Fzero+of%3D%2Ftmp%2Flocal-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e%2Ffile+bs%3D4096+count%3D5120+%26%26+losetup+-f+%2Ftmp%2Flocal-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e%2Ffile&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:02.548: INFO: exec bootstrap-e2e-minion-group-zt76: command: mkdir -p /tmp/local-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e && dd if=/dev/zero of=/tmp/local-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e/file bs=4096 count=5120 && losetup -f /tmp/local-driver-c752e03c-a7a0-4e5e-bf5d-4c0bcf28d17e/file Mar 19 00:31:02.548: INFO: exec bootstrap-e2e-minion-group-zt76: stdout: "" Mar 19 00:31:02.548: INFO: exec bootstrap-e2e-minion-group-zt76: stderr: "" Mar 19 00:31:02.548: INFO: exec bootstrap-e2e-minion-group-zt76: exit code: 0 Mar 19 00:31:02.548: INFO: Unexpected error: <*errors.StatusError | 0xc0007ee460>: error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:135 @ 03/19/23 00:31:02.548 < Exit [It] should support existing directories when readOnly specified in the volumeSource - test/e2e/storage/testsuites/subpath.go:397 @ 03/19/23 00:31:02.548 (11.47s) > Enter [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:02.548 Mar 19 00:31:02.548: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:02.592 (44ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:02.592 STEP: Deleting pod hostexec-bootstrap-e2e-minion-group-zt76-vms47 in namespace provisioning-5171 - test/e2e/framework/pod/delete.go:41 @ 03/19/23 00:31:02.592 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:02.645 (53ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:02.645 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:02.645 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:02.645 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:02.645 STEP: Collecting events from namespace "provisioning-5171". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:02.645 STEP: Found 6 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:02.686 Mar 19 00:31:02.686: INFO: At 2023-03-19 00:30:51 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-vms47: {default-scheduler } Scheduled: Successfully assigned provisioning-5171/hostexec-bootstrap-e2e-minion-group-zt76-vms47 to bootstrap-e2e-minion-group-zt76 Mar 19 00:31:02.686: INFO: At 2023-03-19 00:30:51 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-vms47: {kubelet bootstrap-e2e-minion-group-zt76} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:02.686: INFO: At 2023-03-19 00:30:54 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-vms47: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 3.11692743s (3.11695066s including waiting) Mar 19 00:31:02.686: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-vms47: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container agnhost-container Mar 19 00:31:02.686: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-vms47: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container agnhost-container Mar 19 00:31:02.686: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-vms47: {kubelet bootstrap-e2e-minion-group-zt76} Killing: Stopping container agnhost-container Mar 19 00:31:02.740: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:02.740: INFO: Mar 19 00:31:02.785: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:02.836: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:02.836: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:02.886: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:07.979: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:07.979: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.098: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:08.098: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.203: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.376: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:08.376: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:08.376: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:08.376: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:08.376: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:08.376: INFO: simpletest.rc-pd2gt started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: simpletest.rc-8vdgj started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: simpletest.rc-6gswg started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:08.376: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:08.376: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:08.376: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.376: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.376: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.376: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.376: INFO: simpletest.rc-5vxl5 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: simpletest.rc-4dwp5 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: simpletest.rc-nbwvl started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:08.376: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.376: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:08.376: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:08.376: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:08.376: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.376: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:08.376: INFO: hostexec-bootstrap-e2e-minion-group-494g-jd2vs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.376: INFO: Container agnhost-container ready: false, restart count 0 Mar 19 00:31:08.376: INFO: simpletest.rc-7vn45 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: simpletest.rc-rpvgh started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.376: INFO: simpletest.rc-wzssg started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.361: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:09.361: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:09.457: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 1511 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:09.458: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:09.551: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:09.721: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-w8jmk started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.721: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:09.721: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:09.721: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:09.721: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:09.721: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:09.721: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:09.721: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:09.721: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:09.721: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:09.721: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-xdf8g started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.721: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:09.721: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:09.721: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:09.721: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:09.721: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:09.721: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:09.721: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:09.721: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:09.721: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Init container init-volume-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:09.721: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-69cfd started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.721: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.721: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.721: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.113: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:11.113: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:11.162: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:11.163: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:11.218: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:11.332: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:11.332: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:11.332: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:11.332: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:11.332: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:11.332: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:11.332: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:11.332: INFO: liveness-0d070cba-f6d9-4abd-977a-06769328147f started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:11.332: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:11.332: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:11.332: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:11.332: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:11.332: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:11.332: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:11.332: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container c ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.332: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.332: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:12.598: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:12.598: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:12.695: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:12.695: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:12.832: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:18.106: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-zt76: error trying to reach service: dial tcp 10.138.0.3:10250: i/o timeout END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:18.106 (15.462s) < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:18.106 (15.462s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:18.106 STEP: Destroying namespace "provisioning-5171" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:18.106 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:18.16 (54ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:18.161 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:18.161 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sIn\-tree\sVolumes\s\[Driver\:\slocal\]\[LocalVolumeType\:\sblock\]\s\[Testpattern\:\sPre\-provisioned\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sreadOnly\sfile\sspecified\sin\sthe\svolumeMount\s\[LinuxOnly\]$'
[FAILED] error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:135 @ 03/19/23 00:31:18.585from junit_01.xml
> Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:31:06.609 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:31:06.609 (0s) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:31:06.609 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:31:06.609 Mar 19 00:31:06.609: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/19/23 00:31:06.61 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:31:06.751 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:31:06.832 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:31:06.923 (314ms) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:31:06.923 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:31:06.923 (0s) > Enter [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/19/23 00:31:06.923 Mar 19 00:31:06.973: INFO: In-tree plugin kubernetes.io/local-volume is not migrated, not validating any metrics STEP: Creating block device on node "bootstrap-e2e-minion-group-494g" using path "/tmp/local-driver-e4c69154-9291-4353-8d57-23421619b4d7" - test/e2e/storage/utils/local.go:125 @ 03/19/23 00:31:06.973 Mar 19 00:31:13.410: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-driver-e4c69154-9291-4353-8d57-23421619b4d7 && dd if=/dev/zero of=/tmp/local-driver-e4c69154-9291-4353-8d57-23421619b4d7/file bs=4096 count=5120 && losetup -f /tmp/local-driver-e4c69154-9291-4353-8d57-23421619b4d7/file] Namespace:provisioning-6040 PodName:hostexec-bootstrap-e2e-minion-group-494g-jd2vs ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 19 00:31:13.410: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:31:13.411: INFO: ExecWithOptions: Clientset creation Mar 19 00:31:13.411: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/provisioning-6040/pods/hostexec-bootstrap-e2e-minion-group-494g-jd2vs/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+-p+%2Ftmp%2Flocal-driver-e4c69154-9291-4353-8d57-23421619b4d7+%26%26+dd+if%3D%2Fdev%2Fzero+of%3D%2Ftmp%2Flocal-driver-e4c69154-9291-4353-8d57-23421619b4d7%2Ffile+bs%3D4096+count%3D5120+%26%26+losetup+-f+%2Ftmp%2Flocal-driver-e4c69154-9291-4353-8d57-23421619b4d7%2Ffile&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:18.585: INFO: exec bootstrap-e2e-minion-group-494g: command: mkdir -p /tmp/local-driver-e4c69154-9291-4353-8d57-23421619b4d7 && dd if=/dev/zero of=/tmp/local-driver-e4c69154-9291-4353-8d57-23421619b4d7/file bs=4096 count=5120 && losetup -f /tmp/local-driver-e4c69154-9291-4353-8d57-23421619b4d7/file Mar 19 00:31:18.585: INFO: exec bootstrap-e2e-minion-group-494g: stdout: "" Mar 19 00:31:18.585: INFO: exec bootstrap-e2e-minion-group-494g: stderr: "" Mar 19 00:31:18.585: INFO: exec bootstrap-e2e-minion-group-494g: exit code: 0 Mar 19 00:31:18.585: INFO: Unexpected error: <*errors.StatusError | 0xc0013dac80>: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:135 @ 03/19/23 00:31:18.585 < Exit [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/19/23 00:31:18.585 (11.662s) > Enter [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:18.585 Mar 19 00:31:18.585: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:18.655 (70ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:18.655 STEP: Deleting pod hostexec-bootstrap-e2e-minion-group-494g-jd2vs in namespace provisioning-6040 - test/e2e/framework/pod/delete.go:41 @ 03/19/23 00:31:18.655 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:18.839 (183ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:18.839 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:18.839 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:18.839 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:18.839 STEP: Collecting events from namespace "provisioning-6040". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:18.839 STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:18.908 Mar 19 00:31:18.908: INFO: At 2023-03-19 00:31:07 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-jd2vs: {default-scheduler } Scheduled: Successfully assigned provisioning-6040/hostexec-bootstrap-e2e-minion-group-494g-jd2vs to bootstrap-e2e-minion-group-494g Mar 19 00:31:18.908: INFO: At 2023-03-19 00:31:09 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-jd2vs: {kubelet bootstrap-e2e-minion-group-494g} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 19 00:31:18.908: INFO: At 2023-03-19 00:31:10 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-jd2vs: {kubelet bootstrap-e2e-minion-group-494g} Created: Created container agnhost-container Mar 19 00:31:18.908: INFO: At 2023-03-19 00:31:12 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-jd2vs: {kubelet bootstrap-e2e-minion-group-494g} Started: Started container agnhost-container Mar 19 00:31:18.908: INFO: At 2023-03-19 00:31:18 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-jd2vs: {kubelet bootstrap-e2e-minion-group-494g} Killing: Stopping container agnhost-container Mar 19 00:31:18.964: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:18.964: INFO: Mar 19 00:31:19.012: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:19.059: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:19.059: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:19.109: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:19.219: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container etcd-container ready: true, restart count 0 Mar 19 00:31:19.219: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-19 00:28:29 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 19 00:31:19.219: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-19 00:28:28 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 19 00:31:19.219: INFO: metadata-proxy-v0.1-xbshq started at 2023-03-19 00:28:58 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.219: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:19.219: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:19.219: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 19 00:31:19.219: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container kube-apiserver ready: true, restart count 0 Mar 19 00:31:19.219: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 19 00:31:19.219: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container kube-scheduler ready: true, restart count 0 Mar 19 00:31:19.219: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.219: INFO: Container etcd-container ready: true, restart count 0 Mar 19 00:31:19.581: INFO: Latency metrics for node bootstrap-e2e-master Mar 19 00:31:19.581: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:19.649: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:19.649: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:19.730: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:19.865: INFO: simpletest.rc-x5mkh started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: hostexec-bootstrap-e2e-minion-group-494g-v9nz8 started at 2023-03-19 00:31:16 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-hd6xx started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-vxwr9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: wrong-pod-os started at 2023-03-19 00:31:15 +0000 UTC (0+0 container statuses recorded) Mar 19 00:31:19.865: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-4dwp5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-nbwvl started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-7vn45 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-vbtkv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:19.865: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:19.865: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-slljn started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-gfsns started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.865: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:19.865: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:19.865: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:19.865: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-6gswg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:19.865: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container test-container-subpath-configmap-8sm7 ready: false, restart count 0 Mar 19 00:31:19.865: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-5vxl5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-bgwp2 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-pnjjn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-rpvgh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-wzssg started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-jdnxl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-gps48 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:19.865: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-cn9zr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-pd2gt started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-8vdgj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.865: INFO: simpletest.rc-wpfgd started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.865: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:21.730: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:21.730: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:21.817: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 2324 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:11 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7 kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768 kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768,DevicePath:,},},Config:nil,},} Mar 19 00:31:21.817: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:21.930: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:22.326: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:22.326: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:22.326: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:22.326: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-6x66r started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: pod-55f6ff00-b7bd-449e-a96e-935dccef36ac started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:22.326: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-7sxn7 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-rsmrn started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: pod-7d5a3317-e722-4fa7-8cbf-8ed56da69143 started at 2023-03-19 00:31:11 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:22.326: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:22.326: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:22.326: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:22.326: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:22.326: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-xdf8g started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-5jdgw started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:22.326: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:22.326: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:22.326: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:22.326: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-4l96m started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:22.326: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-b987l started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:22.326: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:22.326: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-69cfd started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-qnn4q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-t66bf started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container agnhost-container ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-w8jmk started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: simpletest.rc-frh8z started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:22.326: INFO: pod-subpath-test-dynamicpv-w4tj started at 2023-03-19 00:31:10 +0000 UTC (2+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Init container init-volume-dynamicpv-w4tj ready: true, restart count 0 Mar 19 00:31:22.326: INFO: Init container test-init-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:22.326: INFO: Container test-container-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:22.326: INFO: httpd-deployment-55bcbdf6f4-9cfj4 started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:22.326: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:26.661: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:26.661: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:26.769: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:26.769: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:26.863: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:27.100: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container write-pod ready: true, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:27.100: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:27.100: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:27.100: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:27.100: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:27.100: INFO: var-expansion-96397faa-9dcb-440b-86ce-3f5b99f6d397 started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container dapi-container ready: false, restart count 0 Mar 19 00:31:27.100: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: pod-client started at 2023-03-19 00:31:13 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container pod-client ready: false, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:27.100: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:27.100: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:27.100: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:27.100: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:27.100: INFO: pod-submit-remove-f472c6a1-a8de-414e-9c1f-f9086d14466f started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container pause ready: false, restart count 0 Mar 19 00:31:27.100: INFO: test-rs-t5n5x started at <nil> (0+0 container statuses recorded) Mar 19 00:31:27.100: INFO: httpd-deployment-55bcbdf6f4-nzqjk started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:27.100: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.100: INFO: Container c ready: false, restart count 0 Mar 19 00:31:27.100: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:27.101: INFO: hostexec-bootstrap-e2e-minion-group-tplt-jfhr8 started at 2023-03-19 00:31:26 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container agnhost-container ready: false, restart count 0 Mar 19 00:31:27.101: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:27.101: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:27.101: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:27.101: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:27.101: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:27.101: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:28.848: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:28.848: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:28.897: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 2650 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:25 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:28.898: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:28.947: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:29.049: INFO: konnectivity-agent-jl7mj started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:29.049: INFO: httpd-deployment-5cd84d4f9-pngsc started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-xdvlv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-vzz8p started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-r4rx7 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: implicit-nonroot-uid started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container implicit-nonroot-uid ready: false, restart count 0 Mar 19 00:31:29.049: INFO: hostexec-bootstrap-e2e-minion-group-zt76-6tdlz started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-pk55q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-zfk5k started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-sck2h started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-cm7k8 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: dns-test-cff8fe3e-c2c5-48b0-8dc1-48e6b5438fcc started at 2023-03-19 00:31:18 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:29.049: INFO: Container jessie-querier ready: false, restart count 0 Mar 19 00:31:29.049: INFO: Container querier ready: false, restart count 0 Mar 19 00:31:29.049: INFO: Container webserver ready: false, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-4m46r started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-j9xqq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-nlqsp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-lss94 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-bpdmm started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-m2wd9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-s7q9z started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: pod-de4e88ee-387a-42e0-a9e4-8a03876dc49f started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container write-pod ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-qz8x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-vpkhp started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: httpd-deployment-55bcbdf6f4-z75nw started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:29.049: INFO: metadata-proxy-v0.1-xn95s started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:29.049: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:29.049: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:29.049: INFO: e2e-net-client started at 2023-03-19 00:31:04 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container e2e-net-client ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-v2h4g started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-qt9cv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-k4vh7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: pvc-tester-n6v47 started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-928rb started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-r77f8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: e2e-net-exec started at 2023-03-19 00:30:58 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 19 00:31:29.049: INFO: ss2-0 started at 2023-03-19 00:31:26 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container webserver ready: false, restart count 0 Mar 19 00:31:29.049: INFO: pod-server-1 started at 2023-03-19 00:31:28 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Init container init ready: false, restart count 0 Mar 19 00:31:29.049: INFO: Container agnhost-container ready: false, restart count 0 Mar 19 00:31:29.049: INFO: kube-proxy-bootstrap-e2e-minion-group-zt76 started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-nb5tk started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-9kxh7 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-nrb9d started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: simpletest.rc-mlpmn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:29.049: INFO: downwardapi-volume-4413fd02-88bd-4a60-b85f-4373b7d12bdc started at 2023-03-19 00:31:25 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:29.049: INFO: Container client-container ready: false, restart count 0 Mar 19 00:31:29.739: INFO: Latency metrics for node bootstrap-e2e-minion-group-zt76 END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:29.739 (10.9s) < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:29.739 (10.9s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:29.739 STEP: Destroying namespace "provisioning-6040" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:29.739 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:29.79 (51ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:29.79 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:29.79 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sIn\-tree\sVolumes\s\[Driver\:\slocal\]\[LocalVolumeType\:\sdir\-bindmounted\]\s\[Testpattern\:\sPre\-provisioned\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sreadOnly\sdirectory\sspecified\sin\sthe\svolumeMount$'
[FAILED] error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:240 @ 03/19/23 00:31:00.596from junit_01.xml
> Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:48.523 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:48.523 (0s) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:48.523 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:48.523 Mar 19 00:30:48.523: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:48.525 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:48.717 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:48.813 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:48.907 (384ms) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.907 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.907 (0s) > Enter [It] should support readOnly directory specified in the volumeMount - test/e2e/storage/testsuites/subpath.go:367 @ 03/19/23 00:30:48.907 Mar 19 00:30:49.005: INFO: In-tree plugin kubernetes.io/local-volume is not migrated, not validating any metrics Mar 19 00:30:55.443: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-driver-d440bd55-4420-4bd2-89da-bd8e013388b8 && mount --bind /tmp/local-driver-d440bd55-4420-4bd2-89da-bd8e013388b8 /tmp/local-driver-d440bd55-4420-4bd2-89da-bd8e013388b8] Namespace:provisioning-8134 PodName:hostexec-bootstrap-e2e-minion-group-tplt-q86pt ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 19 00:30:55.443: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:30:55.444: INFO: ExecWithOptions: Clientset creation Mar 19 00:30:55.444: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/provisioning-8134/pods/hostexec-bootstrap-e2e-minion-group-tplt-q86pt/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+%2Ftmp%2Flocal-driver-d440bd55-4420-4bd2-89da-bd8e013388b8+%26%26+mount+--bind+%2Ftmp%2Flocal-driver-d440bd55-4420-4bd2-89da-bd8e013388b8+%2Ftmp%2Flocal-driver-d440bd55-4420-4bd2-89da-bd8e013388b8&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:00.595: INFO: exec bootstrap-e2e-minion-group-tplt: command: mkdir /tmp/local-driver-d440bd55-4420-4bd2-89da-bd8e013388b8 && mount --bind /tmp/local-driver-d440bd55-4420-4bd2-89da-bd8e013388b8 /tmp/local-driver-d440bd55-4420-4bd2-89da-bd8e013388b8 Mar 19 00:31:00.595: INFO: exec bootstrap-e2e-minion-group-tplt: stdout: "" Mar 19 00:31:00.595: INFO: exec bootstrap-e2e-minion-group-tplt: stderr: "" Mar 19 00:31:00.595: INFO: exec bootstrap-e2e-minion-group-tplt: exit code: 0 Mar 19 00:31:00.595: INFO: Unexpected error: <*errors.StatusError | 0xc000142fa0>: error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:240 @ 03/19/23 00:31:00.596 < Exit [It] should support readOnly directory specified in the volumeMount - test/e2e/storage/testsuites/subpath.go:367 @ 03/19/23 00:31:00.596 (11.688s) > Enter [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:00.596 Mar 19 00:31:00.596: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:00.646 (51ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:00.646 STEP: Deleting pod hostexec-bootstrap-e2e-minion-group-tplt-q86pt in namespace provisioning-8134 - test/e2e/framework/pod/delete.go:41 @ 03/19/23 00:31:00.646 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:00.76 (113ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:00.76 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:00.76 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:00.76 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:00.76 STEP: Collecting events from namespace "provisioning-8134". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:00.76 STEP: Found 6 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:00.801 Mar 19 00:31:00.801: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-q86pt: {default-scheduler } Scheduled: Successfully assigned provisioning-8134/hostexec-bootstrap-e2e-minion-group-tplt-q86pt to bootstrap-e2e-minion-group-tplt Mar 19 00:31:00.801: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-q86pt: {kubelet bootstrap-e2e-minion-group-tplt} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:00.801: INFO: At 2023-03-19 00:30:53 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-q86pt: {kubelet bootstrap-e2e-minion-group-tplt} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 3.316319545s (3.316337025s including waiting) Mar 19 00:31:00.801: INFO: At 2023-03-19 00:30:53 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-q86pt: {kubelet bootstrap-e2e-minion-group-tplt} Created: Created container agnhost-container Mar 19 00:31:00.801: INFO: At 2023-03-19 00:30:53 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-q86pt: {kubelet bootstrap-e2e-minion-group-tplt} Started: Started container agnhost-container Mar 19 00:31:00.801: INFO: At 2023-03-19 00:31:00 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-q86pt: {kubelet bootstrap-e2e-minion-group-tplt} Killing: Stopping container agnhost-container Mar 19 00:31:00.842: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:00.842: INFO: Mar 19 00:31:00.887: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:00.929: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:00.929: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:00.972: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:06.017: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:06.017: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.070: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:06.070: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.113: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.167: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:06.167: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:06.167: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:06.167: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:06.167: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:06.167: INFO: e2e-dns-utils started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:06.167: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:06.167: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:06.167: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:06.167: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:06.167: INFO: e2e-test-httpd-pod started at 2023-03-19 00:30:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container e2e-test-httpd-pod ready: false, restart count 1 Mar 19 00:31:06.167: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:06.167: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:06.167: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:06.167: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:06.167: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:06.167: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.167: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:06.394: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.394: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:06.469: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 1511 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:06.469: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:06.511: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:06.576: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.576: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:06.576: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.576: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:06.576: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.576: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:06.576: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.576: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:06.576: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:06.577: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:06.577: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:06.577: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:06.577: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:06.577: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.577: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:06.577: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:06.577: INFO: Init container init-volume-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:06.577: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.577: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:06.577: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.577: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:06.577: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:06.577: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:06.577: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:06.577: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:06.577: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:06.577: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:06.577: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:06.577: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:06.577: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.577: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:06.577: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.577: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:06.871: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:06.871: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:06.918: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1643 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:56 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:56 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:56 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:56 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:06.918: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:06.964: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:07.073: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:07.073: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:07.073: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:07.073: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:07.073: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:07.073: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:07.073: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:07.073: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:07.073: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:07.073: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:07.073: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:07.073: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:07.073: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:07.073: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:07.073: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:07.073: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:07.073: INFO: liveness-0d070cba-f6d9-4abd-977a-06769328147f started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:07.073: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:07.073: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.073: INFO: Container c ready: false, restart count 0 Mar 19 00:31:07.743: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:07.743: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:07.872: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:07.872: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:08.066: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:13.213: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-zt76: error trying to reach service: dial tcp 10.138.0.3:10250: i/o timeout END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:13.213 (12.453s) < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:13.213 (12.453s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:13.213 STEP: Destroying namespace "provisioning-8134" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:13.213 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:13.278 (65ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:13.278 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:13.278 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sIn\-tree\sVolumes\s\[Driver\:\slocal\]\[LocalVolumeType\:\sdir\-link\]\s\[Testpattern\:\sPre\-provisioned\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sreadOnly\sfile\sspecified\sin\sthe\svolumeMount\s\[LinuxOnly\]$'
[FAILED] error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:220 @ 03/19/23 00:31:02.164from junit_01.xml
> Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:50.445 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:50.445 (0s) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:50.445 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:50.445 Mar 19 00:30:50.445: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:50.446 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:50.581 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:50.662 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:50.742 (297ms) > Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:50.742 < Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:50.742 (0s) > Enter [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/19/23 00:30:50.742 Mar 19 00:30:50.789: INFO: In-tree plugin kubernetes.io/local-volume is not migrated, not validating any metrics Mar 19 00:30:57.012: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67-backend && ln -s /tmp/local-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67-backend /tmp/local-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67] Namespace:provisioning-7200 PodName:hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 19 00:30:57.012: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:30:57.013: INFO: ExecWithOptions: Clientset creation Mar 19 00:30:57.013: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/provisioning-7200/pods/hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+%2Ftmp%2Flocal-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67-backend+%26%26+ln+-s+%2Ftmp%2Flocal-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67-backend+%2Ftmp%2Flocal-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:02.164: INFO: exec bootstrap-e2e-minion-group-g5rl: command: mkdir /tmp/local-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67-backend && ln -s /tmp/local-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67-backend /tmp/local-driver-dbf05832-6be9-4112-b6cd-e493cb97fa67 Mar 19 00:31:02.164: INFO: exec bootstrap-e2e-minion-group-g5rl: stdout: "" Mar 19 00:31:02.164: INFO: exec bootstrap-e2e-minion-group-g5rl: stderr: "" Mar 19 00:31:02.164: INFO: exec bootstrap-e2e-minion-group-g5rl: exit code: 0 Mar 19 00:31:02.164: INFO: Unexpected error: <*errors.StatusError | 0xc000e9e460>: error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout In [It] at: test/e2e/storage/utils/local.go:220 @ 03/19/23 00:31:02.164 < Exit [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/19/23 00:31:02.164 (11.422s) > Enter [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:02.164 Mar 19 00:31:02.164: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:02.215 (51ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:02.215 STEP: Deleting pod hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2 in namespace provisioning-7200 - test/e2e/framework/pod/delete.go:41 @ 03/19/23 00:31:02.215 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:02.268 (52ms) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:02.268 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:02.268 (0s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:02.268 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:02.268 STEP: Collecting events from namespace "provisioning-7200". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:02.268 STEP: Found 6 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:02.31 Mar 19 00:31:02.310: INFO: At 2023-03-19 00:30:50 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2: {default-scheduler } Scheduled: Successfully assigned provisioning-7200/hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2 to bootstrap-e2e-minion-group-g5rl Mar 19 00:31:02.310: INFO: At 2023-03-19 00:30:51 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2: {kubelet bootstrap-e2e-minion-group-g5rl} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:02.310: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2: {kubelet bootstrap-e2e-minion-group-g5rl} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 4.489446932s (4.489463123s including waiting) Mar 19 00:31:02.310: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2: {kubelet bootstrap-e2e-minion-group-g5rl} Created: Created container agnhost-container Mar 19 00:31:02.310: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2: {kubelet bootstrap-e2e-minion-group-g5rl} Started: Started container agnhost-container Mar 19 00:31:02.310: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-jgdg2: {kubelet bootstrap-e2e-minion-group-g5rl} Killing: Stopping container agnhost-container Mar 19 00:31:02.351: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:02.351: INFO: Mar 19 00:31:02.401: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:02.472: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:02.472: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:02.543: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:07.636: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:07.636: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:07.805: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:07.806: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:07.979: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.224: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:08.224: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:08.224: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:08.224: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:08.224: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:08.224: INFO: simpletest.rc-pd2gt started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.224: INFO: simpletest.rc-8vdgj started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.224: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:08.224: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:08.224: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:08.224: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.224: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.224: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.224: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.224: INFO: simpletest.rc-5vxl5 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.224: INFO: simpletest.rc-4dwp5 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.224: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:08.224: INFO: simpletest.rc-l8ssf started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.224: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:08.224: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:08.224: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:08.224: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.224: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:08.224: INFO: hostexec-bootstrap-e2e-minion-group-494g-jd2vs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.224: INFO: Container agnhost-container ready: false, restart count 0 Mar 19 00:31:08.224: INFO: simpletest.rc-7vn45 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.224: INFO: simpletest.rc-rpvgh started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.224: INFO: simpletest.rc-wzssg started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.701: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:08.701: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:08.879: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 1511 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:08.880: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:09.030: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:09.234: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-szxkk started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.234: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:09.234: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:09.234: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:09.234: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:09.234: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:09.234: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:09.234: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:09.234: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:09.234: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:09.234: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-6m7f4 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.234: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:09.234: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:09.234: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:09.234: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-jzxp6 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.234: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:09.234: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.234: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:09.234: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:09.234: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:09.234: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:09.234: INFO: Init container init-volume-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:09.234: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:09.906: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:09.906: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:09.984: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:09.985: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:10.058: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:10.244: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.244: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:10.244: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.244: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.244: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.244: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:10.244: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.244: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.244: INFO: simpletest.rc-zvjzr started at <nil> (0+0 container statuses recorded) Mar 19 00:31:10.244: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:10.245: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:10.245: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:10.245: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:10.245: INFO: liveness-0d070cba-f6d9-4abd-977a-06769328147f started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:10.245: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:10.245: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:10.245: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:10.245: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:10.245: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container c ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-4fqhh started at <nil> (0+0 container statuses recorded) Mar 19 00:31:10.245: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:10.245: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:10.245: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:10.245: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-kltzl started at <nil> (0+0 container statuses recorded) Mar 19 00:31:10.245: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:10.245: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:10.245: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:12.594: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:12.594: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:12.714: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:12.714: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:12.911: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:18.151: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-zt76: error trying to reach service: dial tcp 10.138.0.3:10250: i/o timeout END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:18.151 (15.883s) < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:18.151 (15.883s) > Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:18.151 STEP: Destroying namespace "provisioning-7200" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:18.151 < Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:18.194 (43ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:18.194 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:18.194 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sIn\-tree\sVolumes\s\[Driver\:\slocal\]\[LocalVolumeType\:\stmpfs\]\s\[Testpattern\:\sPre\-provisioned\sPV\s\(default\sfs\)\]\ssubPath\sshould\ssupport\sreadOnly\sfile\sspecified\sin\sthe\svolumeMount\s\[LinuxOnly\]$'
[FAILED] failed to get logs from pod-subpath-test-preprovisionedpv-4nr4 for test-container-subpath-preprovisionedpv-4nr4: an error on the server ("unknown") has prevented the request from succeeding (get pods pod-subpath-test-preprovisionedpv-4nr4)
In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/19/23 00:31:13.356
There were additional failures detected after the initial failure. These are visible in the timeline
> Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:47.434
< Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/framework/testsuite.go:51 @ 03/19/23 00:30:47.435 (0s)
> Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:47.435
STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:47.435
Mar 19 00:30:47.435: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename provisioning - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:47.437
STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:47.824
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:48.018
< Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - set up framework | framework.go:191 @ 03/19/23 00:30:48.217 (783ms)
> Enter [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.217
< Exit [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.217 (0s)
> Enter [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/19/23 00:30:48.217
Mar 19 00:30:48.418: INFO: In-tree plugin kubernetes.io/local-volume is not migrated, not validating any metrics
STEP: Creating tmpfs mount point on node "bootstrap-e2e-minion-group-zt76" at path "/tmp/local-driver-b57a9e8a-d278-4da3-893b-31c670141421" - test/e2e/storage/utils/local.go:104 @ 03/19/23 00:30:48.418
Mar 19 00:30:56.792: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p "/tmp/local-driver-b57a9e8a-d278-4da3-893b-31c670141421" && mount -t tmpfs -o size=10m tmpfs-"/tmp/local-driver-b57a9e8a-d278-4da3-893b-31c670141421" "/tmp/local-driver-b57a9e8a-d278-4da3-893b-31c670141421"] Namespace:provisioning-3638 PodName:hostexec-bootstrap-e2e-minion-group-zt76-6t6wj ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false}
Mar 19 00:30:56.792: INFO: >>> kubeConfig: /workspace/.kube/config
Mar 19 00:30:56.793: INFO: ExecWithOptions: Clientset creation
Mar 19 00:30:56.793: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/provisioning-3638/pods/hostexec-bootstrap-e2e-minion-group-zt76-6t6wj/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+-p+%22%2Ftmp%2Flocal-driver-b57a9e8a-d278-4da3-893b-31c670141421%22+%26%26+mount+-t+tmpfs+-o+size%3D10m+tmpfs-%22%2Ftmp%2Flocal-driver-b57a9e8a-d278-4da3-893b-31c670141421%22+%22%2Ftmp%2Flocal-driver-b57a9e8a-d278-4da3-893b-31c670141421%22&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true)
Mar 19 00:30:57.276: INFO: Creating resource for pre-provisioned PV
Mar 19 00:30:57.276: INFO: Creating PVC and PV
STEP: Creating a PVC followed by a PV - test/e2e/framework/pv/pv.go:360 @ 03/19/23 00:30:57.276
Mar 19 00:30:57.410: INFO: Waiting for PV local-rgjvp to bind to PVC pvc-6pq2v
Mar 19 00:30:57.410: INFO: Waiting up to timeout=3m0s for PersistentVolumeClaims [pvc-6pq2v] to have phase Bound
Mar 19 00:30:57.453: INFO: PersistentVolumeClaim pvc-6pq2v found but phase is Pending instead of Bound.
Mar 19 00:30:59.495: INFO: PersistentVolumeClaim pvc-6pq2v found but phase is Pending instead of Bound.
Mar 19 00:31:01.542: INFO: PersistentVolumeClaim pvc-6pq2v found and phase=Bound (4.131449842s)
Mar 19 00:31:01.542: INFO: Waiting up to 3m0s for PersistentVolume local-rgjvp to have phase Bound
Mar 19 00:31:01.583: INFO: PersistentVolume local-rgjvp found and phase=Bound (41.499035ms)
STEP: Creating pod pod-subpath-test-preprovisionedpv-4nr4 - test/e2e/storage/testsuites/subpath.go:691 @ 03/19/23 00:31:01.686
STEP: Creating a pod to test subpath - test/e2e/framework/pod/output/output.go:243 @ 03/19/23 00:31:01.686
STEP: Saw pod success - test/e2e/framework/pod/wait.go:409 @ 03/19/23 00:31:08.026
Mar 19 00:31:08.110: INFO: Trying to get logs from node bootstrap-e2e-minion-group-zt76 pod pod-subpath-test-preprovisionedpv-4nr4 container test-container-subpath-preprovisionedpv-4nr4: <nil>
Mar 19 00:31:13.264: INFO: Failed to get logs from node "bootstrap-e2e-minion-group-zt76" pod "pod-subpath-test-preprovisionedpv-4nr4" container "test-container-subpath-preprovisionedpv-4nr4". an error on the server ("unknown") has prevented the request from succeeding (get pods pod-subpath-test-preprovisionedpv-4nr4)
STEP: delete the pod - test/e2e/framework/pod/output/output.go:168 @ 03/19/23 00:31:13.264
Mar 19 00:31:13.355: INFO: Unexpected error:
<*fmt.wrapError | 0xc006092560>:
failed to get logs from pod-subpath-test-preprovisionedpv-4nr4 for test-container-subpath-preprovisionedpv-4nr4: an error on the server ("unknown") has prevented the request from succeeding (get pods pod-subpath-test-preprovisionedpv-4nr4)
{
msg: "failed to get logs from pod-subpath-test-preprovisionedpv-4nr4 for test-container-subpath-preprovisionedpv-4nr4: an error on the server (\"unknown\") has prevented the request from succeeding (get pods pod-subpath-test-preprovisionedpv-4nr4)",
err: <*errors.StatusError | 0xc0006af540>{
ErrStatus: {
TypeMeta: {Kind: "", APIVersion: ""},
ListMeta: {
SelfLink: "",
ResourceVersion: "",
Continue: "",
RemainingItemCount: nil,
},
Status: "Failure",
Message: "an error on the server (\"unknown\") has prevented the request from succeeding (get pods pod-subpath-test-preprovisionedpv-4nr4)",
Reason: "InternalError",
Details: {
Name: "pod-subpath-test-preprovisionedpv-4nr4",
Group: "",
Kind: "pods",
UID: "",
Causes: [
{
Type: "UnexpectedServerResponse",
Message: "unknown",
Field: "",
},
],
RetryAfterSeconds: 0,
},
Code: 500,
},
},
}
[FAILED] failed to get logs from pod-subpath-test-preprovisionedpv-4nr4 for test-container-subpath-preprovisionedpv-4nr4: an error on the server ("unknown") has prevented the request from succeeding (get pods pod-subpath-test-preprovisionedpv-4nr4)
In [It] at: test/e2e/framework/pod/output/output.go:247 @ 03/19/23 00:31:13.356
< Exit [It] should support readOnly file specified in the volumeMount [LinuxOnly] - test/e2e/storage/testsuites/subpath.go:382 @ 03/19/23 00:31:13.356 (25.139s)
> Enter [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:13.356
Mar 19 00:31:13.356: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
< Exit [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:13.438 (82ms)
> Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:384 @ 03/19/23 00:31:13.438
STEP: Deleting pod - test/e2e/storage/testsuites/subpath.go:172 @ 03/19/23 00:31:13.438
Mar 19 00:31:13.438: INFO: Deleting pod "pod-subpath-test-preprovisionedpv-4nr4" in namespace "provisioning-3638"
STEP: Deleting pv and pvc - test/e2e/storage/framework/volume_resource.go:176 @ 03/19/23 00:31:13.48
Mar 19 00:31:13.480: INFO: Deleting PersistentVolumeClaim "pvc-6pq2v"
Mar 19 00:31:13.527: INFO: Deleting PersistentVolume "local-rgjvp"
STEP: Unmount tmpfs mount point on node "bootstrap-e2e-minion-group-zt76" at path "/tmp/local-driver-b57a9e8a-d278-4da3-893b-31c670141421" - test/e2e/storage/utils/local.go:114 @ 03/19/23 00:31:13.576
Mar 19 00:31:13.576: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c umount "/tmp/local-driver-b57a9e8a-d278-4da3-893b-31c670141421"] Namespace:provisioning-3638 PodName:hostexec-bootstrap-e2e-minion-group-zt76-6t6wj ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false}
Mar 19 00:31:13.576: INFO: >>> kubeConfig: /workspace/.kube/config
Mar 19 00:31:13.577: INFO: ExecWithOptions: Clientset creation
Mar 19 00:31:13.577: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/provisioning-3638/pods/hostexec-bootstrap-e2e-minion-group-zt76-6t6wj/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=umount+%22%2Ftmp%2Flocal-driver-b57a9e8a-d278-4da3-893b-31c670141421%22&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true)
Mar 19 00:31:18.739: INFO: exec bootstrap-e2e-minion-group-zt76: command: umount "/tmp/local-driver-b57a9e8a-d278-4da3-893b-31c670141421"
Mar 19 00:31:18.739: INFO: exec bootstrap-e2e-minion-group-zt76: stdout: ""
Mar 19 00:31:18.739: INFO: exec bootstrap-e2e-minion-group-zt76: stderr: ""
Mar 19 00:31:18.739: INFO: exec bootstrap-e2e-minion-group-zt76: exit code: 0
Mar 19 00:31:18.739: INFO: Unexpected error:
<*errors.StatusError | 0xc00051e0a0>:
error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout
{
ErrStatus:
code: 500
message: 'error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout'
metadata: {}
status: Failure,
}
[FAILED] error dialing backend: dial tcp 10.138.0.3:10250: i/o timeout
In [DeferCleanup (Each)] at: test/e2e/storage/utils/local.go:116 @ 03/19/23 00:31:18.739
< Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/testsuites/subpath.go:384 @ 03/19/23 00:31:18.739 (5.302s)
> Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:18.739
STEP: Deleting pod hostexec-bootstrap-e2e-minion-group-zt76-6t6wj in namespace provisioning-3638 - test/e2e/framework/pod/delete.go:41 @ 03/19/23 00:31:18.739
< Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/storage/drivers/in_tree.go:1664 @ 03/19/23 00:31:18.878 (139ms)
> Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:18.878
< Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:18.878 (0s)
> Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:18.878
STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:18.878
STEP: Collecting events from namespace "provisioning-3638". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:18.878
STEP: Found 17 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:18.953
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:30:48 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-6t6wj: {default-scheduler } Scheduled: Successfully assigned provisioning-3638/hostexec-bootstrap-e2e-minion-group-zt76-6t6wj to bootstrap-e2e-minion-group-zt76
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-6t6wj: {kubelet bootstrap-e2e-minion-group-zt76} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43"
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:30:54 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-6t6wj: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 5.697540287s (5.697555533s including waiting)
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-6t6wj: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container agnhost-container
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-6t6wj: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container agnhost-container
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:30:57 +0000 UTC - event for pvc-6pq2v: {persistentvolume-controller } ProvisioningFailed: storageclass.storage.k8s.io "provisioning-3638" not found
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:01 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {default-scheduler } Scheduled: Successfully assigned provisioning-3638/pod-subpath-test-preprovisionedpv-4nr4 to bootstrap-e2e-minion-group-zt76
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container init-volume-preprovisionedpv-4nr4
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container init-volume-preprovisionedpv-4nr4
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:02 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container test-init-volume-preprovisionedpv-4nr4
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:03 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container test-init-volume-preprovisionedpv-4nr4
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:03 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:03 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Created: Created container test-container-subpath-preprovisionedpv-4nr4
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:04 +0000 UTC - event for pod-subpath-test-preprovisionedpv-4nr4: {kubelet bootstrap-e2e-minion-group-zt76} Started: Started container test-container-subpath-preprovisionedpv-4nr4
Mar 19 00:31:18.953: INFO: At 2023-03-19 00:31:18 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-zt76-6t6wj: {kubelet bootstrap-e2e-minion-group-zt76} Killing: Stopping container agnhost-container
Mar 19 00:31:19.004: INFO: POD NODE PHASE GRACE CONDITIONS
Mar 19 00:31:19.004: INFO:
Mar 19 00:31:19.056: INFO:
Logging node info for node bootstrap-e2e-master
Mar 19 00:31:19.103: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Mar 19 00:31:19.104: INFO:
Logging kubelet events for node bootstrap-e2e-master
Mar 19 00:31:19.156: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Mar 19 00:31:19.244: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-19 00:28:28 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container kube-addon-manager ready: true, restart count 0
Mar 19 00:31:19.244: INFO: metadata-proxy-v0.1-xbshq started at 2023-03-19 00:28:58 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:19.244: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:19.244: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container konnectivity-server-container ready: true, restart count 0
Mar 19 00:31:19.244: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container kube-apiserver ready: true, restart count 0
Mar 19 00:31:19.244: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container kube-controller-manager ready: true, restart count 1
Mar 19 00:31:19.244: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container kube-scheduler ready: true, restart count 0
Mar 19 00:31:19.244: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container etcd-container ready: true, restart count 0
Mar 19 00:31:19.244: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-19 00:28:12 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container etcd-container ready: true, restart count 0
Mar 19 00:31:19.244: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-19 00:28:29 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:19.244: INFO: Container l7-lb-controller ready: true, restart count 2
Mar 19 00:31:19.580: INFO:
Latency metrics for node bootstrap-e2e-master
Mar 19 00:31:19.580: INFO:
Logging node info for node bootstrap-e2e-minion-group-494g
Mar 19 00:31:19.645: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Mar 19 00:31:19.645: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-494g
Mar 19 00:31:19.720: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g
Mar 19 00:31:20.019: INFO: simpletest.rc-gps48 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:20.019: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container agnhost ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-rpvgh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-wzssg started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-jdnxl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-pd2gt started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-8vdgj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-wpfgd started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-cn9zr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-hd6xx started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-x5mkh started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: hostexec-bootstrap-e2e-minion-group-494g-v9nz8 started at 2023-03-19 00:31:16 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:20.019: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-4dwp5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-nbwvl started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-vxwr9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: wrong-pod-os started at 2023-03-19 00:31:15 +0000 UTC (0+0 container statuses recorded)
Mar 19 00:31:20.019: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:20.019: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container coredns ready: true, restart count 0
Mar 19 00:31:20.019: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nfs-server ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-7vn45 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-vbtkv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:20.019: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:20.019: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0
Mar 19 00:31:20.019: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-6gswg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-slljn started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-gfsns started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:20.019: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container test-container-subpath-configmap-8sm7 ready: false, restart count 0
Mar 19 00:31:20.019: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container e2e-net-server ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-5vxl5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-bgwp2 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:20.019: INFO: simpletest.rc-pnjjn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:20.019: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:21.735: INFO:
Latency metrics for node bootstrap-e2e-minion-group-494g
Mar 19 00:31:21.735: INFO:
Logging node info for node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:21.818: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 2324 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:11 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:16 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:16 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7 kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768 kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-6864^57a25e99-c5ed-11ed-875f-760b3219ffc7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-62^57a67a98-c5ed-11ed-aef1-9e7790c5d6de,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-read-write-once-pod-5648^5797e429-c5ed-11ed-87b1-1e84fe2ab768,DevicePath:,},},Config:nil,},}
Mar 19 00:31:21.818: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:21.903: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:22.329: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-69cfd started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container portforwardtester ready: true, restart count 0
Mar 19 00:31:22.329: INFO: Container readiness ready: true, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-qnn4q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-t66bf started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container agnhost-container ready: false, restart count 0
Mar 19 00:31:22.329: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container autoscaler ready: true, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-w8jmk started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-frh8z started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: pod-subpath-test-dynamicpv-w4tj started at 2023-03-19 00:31:10 +0000 UTC (2+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Init container init-volume-dynamicpv-w4tj ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Init container test-init-subpath-dynamicpv-w4tj ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container test-container-subpath-dynamicpv-w4tj ready: false, restart count 0
Mar 19 00:31:22.329: INFO: httpd-deployment-55bcbdf6f4-9cfj4 started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container httpd ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container csi-attacher ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-provisioner ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-resizer ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-snapshotter ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container hostpath ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container liveness-probe ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container node-driver-registrar ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-6x66r started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: pod-55f6ff00-b7bd-449e-a96e-935dccef36ac started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container write-pod ready: false, restart count 0
Mar 19 00:31:22.329: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container coredns ready: true, restart count 0
Mar 19 00:31:22.329: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-rsmrn started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container pod-csi-inline-volumes ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-7sxn7 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container csi-attacher ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-provisioner ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-resizer ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-snapshotter ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container hostpath ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container liveness-probe ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container node-driver-registrar ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-xdf8g started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-5jdgw started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: pod-7d5a3317-e722-4fa7-8cbf-8ed56da69143 started at 2023-03-19 00:31:11 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container write-pod ready: false, restart count 0
Mar 19 00:31:22.329: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:22.329: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container csi-attacher ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-provisioner ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-resizer ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container csi-snapshotter ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container hostpath ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container liveness-probe ready: false, restart count 0
Mar 19 00:31:22.329: INFO: Container node-driver-registrar ready: false, restart count 0
Mar 19 00:31:22.329: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:22.329: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:22.329: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nfs-server ready: true, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-b987l started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: simpletest.rc-4l96m started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:22.329: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container volume-snapshot-controller ready: true, restart count 0
Mar 19 00:31:22.329: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:22.329: INFO: Container default-http-backend ready: true, restart count 0
Mar 19 00:31:26.371: INFO:
Latency metrics for node bootstrap-e2e-minion-group-g5rl
Mar 19 00:31:26.371: INFO:
Logging node info for node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:26.482: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},}
Mar 19 00:31:26.482: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:26.566: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:26.753: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: pod-submit-remove-f472c6a1-a8de-414e-9c1f-f9086d14466f started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container pause ready: false, restart count 0
Mar 19 00:31:26.753: INFO: test-rs-t5n5x started at <nil> (0+0 container statuses recorded)
Mar 19 00:31:26.753: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container c ready: false, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: httpd-deployment-55bcbdf6f4-nzqjk started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container httpd ready: false, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: hostexec-bootstrap-e2e-minion-group-tplt-jfhr8 started at 2023-03-19 00:31:26 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container agnhost-container ready: false, restart count 0
Mar 19 00:31:26.753: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container csi-provisioner ready: true, restart count 0
Mar 19 00:31:26.753: INFO: Container driver-registrar ready: true, restart count 0
Mar 19 00:31:26.753: INFO: Container mock ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container write-pod ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: false, restart count 0
Mar 19 00:31:26.753: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:26.753: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container metrics-server ready: true, restart count 0
Mar 19 00:31:26.753: INFO: Container metrics-server-nanny ready: true, restart count 0
Mar 19 00:31:26.753: INFO: var-expansion-96397faa-9dcb-440b-86ce-3f5b99f6d397 started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container dapi-container ready: false, restart count 0
Mar 19 00:31:26.753: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container csi-attacher ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: pod-client started at 2023-03-19 00:31:13 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container pod-client ready: false, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:26.753: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:26.753: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:26.753: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:26.753: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0
Mar 19 00:31:26.753: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:26.753: INFO: Container volume-tester ready: true, restart count 0
Mar 19 00:31:29.121: INFO:
Latency metrics for node bootstrap-e2e-minion-group-tplt
Mar 19 00:31:29.121: INFO:
Logging node info for node bootstrap-e2e-minion-group-zt76
Mar 19 00:31:29.166: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 2650 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:25 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:25 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Mar 19 00:31:29.166: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-zt76
Mar 19 00:31:29.213: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76
Mar 19 00:31:29.280: INFO: e2e-net-client started at 2023-03-19 00:31:04 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container e2e-net-client ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-v2h4g started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-qt9cv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-k4vh7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: metadata-proxy-v0.1-xn95s started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container metadata-proxy ready: true, restart count 0
Mar 19 00:31:29.280: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-928rb started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-r77f8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: pvc-tester-n6v47 started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container write-pod ready: false, restart count 0
Mar 19 00:31:29.280: INFO: e2e-net-exec started at 2023-03-19 00:30:58 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container e2e-net-exec ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-nb5tk started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-9kxh7 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-nrb9d started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-mlpmn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: downwardapi-volume-4413fd02-88bd-4a60-b85f-4373b7d12bdc started at 2023-03-19 00:31:25 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container client-container ready: false, restart count 0
Mar 19 00:31:29.280: INFO: ss2-0 started at 2023-03-19 00:31:26 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container webserver ready: false, restart count 0
Mar 19 00:31:29.280: INFO: pod-server-1 started at 2023-03-19 00:31:28 +0000 UTC (1+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Init container init ready: false, restart count 0
Mar 19 00:31:29.280: INFO: Container agnhost-container ready: false, restart count 0
Mar 19 00:31:29.280: INFO: kube-proxy-bootstrap-e2e-minion-group-zt76 started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container kube-proxy ready: true, restart count 0
Mar 19 00:31:29.280: INFO: httpd-deployment-5cd84d4f9-pngsc started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container httpd ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-xdvlv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-vzz8p started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-r4rx7 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: implicit-nonroot-uid started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container implicit-nonroot-uid ready: false, restart count 0
Mar 19 00:31:29.280: INFO: konnectivity-agent-jl7mj started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container konnectivity-agent ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-pk55q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-zfk5k started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-sck2h started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: hostexec-bootstrap-e2e-minion-group-zt76-6tdlz started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container agnhost-container ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-j9xqq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-nlqsp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-lss94 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-bpdmm started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-m2wd9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-cm7k8 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: dns-test-cff8fe3e-c2c5-48b0-8dc1-48e6b5438fcc started at 2023-03-19 00:31:18 +0000 UTC (0+3 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container jessie-querier ready: false, restart count 0
Mar 19 00:31:29.280: INFO: Container querier ready: false, restart count 0
Mar 19 00:31:29.280: INFO: Container webserver ready: false, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-4m46r started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: pod-de4e88ee-387a-42e0-a9e4-8a03876dc49f started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container write-pod ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-qz8x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-vpkhp started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:29.280: INFO: httpd-deployment-55bcbdf6f4-z75nw started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container httpd ready: false, restart count 0
Mar 19 00:31:29.280: INFO: simpletest.rc-s7q9z started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded)
Mar 19 00:31:29.280: INFO: Container nginx ready: true, restart count 0
Mar 19 00:31:31.032: INFO:
Latency metrics for node bootstrap-e2e-minion-group-zt76
END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:31.032 (12.154s)
< Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - dump namespaces | framework.go:209 @ 03/19/23 00:31:31.032 (12.154s)
> Enter [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:31.032
STEP: Destroying namespace "provisioning-3638" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:31.032
< Exit [DeferCleanup (Each)] [Testpattern: Pre-provisioned PV (default fs)] subPath - tear down framework | framework.go:206 @ 03/19/23 00:31:31.092 (60ms)
> Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:31.093
< Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:31.093 (0s)
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sPersistentVolumes\-local\s\sPod\swith\snode\sdifferent\sfrom\sPV\'s\sNodeAffinity\sshould\sfail\sscheduling\sdue\sto\sdifferent\sNodeSelector$'
[FAILED] error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout In [BeforeEach] at: test/e2e/storage/utils/local.go:201 @ 03/19/23 00:31:01.099 There were additional failures detected after the initial failure. These are visible in the timelinefrom junit_01.xml
> Enter [BeforeEach] [sig-storage] PersistentVolumes-local - set up framework | framework.go:191 @ 03/19/23 00:30:46.52 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:46.52 Mar 19 00:30:46.520: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:46.521 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:46.817 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:47.017 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - set up framework | framework.go:191 @ 03/19/23 00:30:47.219 (699ms) > Enter [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:47.219 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:47.219 (0s) > Enter [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/storage/persistent_volumes-local.go:161 @ 03/19/23 00:30:47.219 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/storage/persistent_volumes-local.go:161 @ 03/19/23 00:30:47.418 (198ms) > Enter [BeforeEach] Pod with node different from PV's NodeAffinity - test/e2e/storage/persistent_volumes-local.go:365 @ 03/19/23 00:30:47.418 STEP: Initializing test volumes - test/e2e/storage/persistent_volumes-local.go:1117 @ 03/19/23 00:30:47.517 Mar 19 00:30:55.958: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-873c92d6-9a0c-48fd-8d44-95b6fb4e0ad5] Namespace:persistent-local-volumes-test-8935 PodName:hostexec-bootstrap-e2e-minion-group-tplt-shjgn ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 19 00:30:55.958: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:30:55.959: INFO: ExecWithOptions: Clientset creation Mar 19 00:30:55.959: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/persistent-local-volumes-test-8935/pods/hostexec-bootstrap-e2e-minion-group-tplt-shjgn/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+-p+%2Ftmp%2Flocal-volume-test-873c92d6-9a0c-48fd-8d44-95b6fb4e0ad5&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:01.098: INFO: exec bootstrap-e2e-minion-group-tplt: command: mkdir -p /tmp/local-volume-test-873c92d6-9a0c-48fd-8d44-95b6fb4e0ad5 Mar 19 00:31:01.098: INFO: exec bootstrap-e2e-minion-group-tplt: stdout: "" Mar 19 00:31:01.098: INFO: exec bootstrap-e2e-minion-group-tplt: stderr: "" Mar 19 00:31:01.098: INFO: exec bootstrap-e2e-minion-group-tplt: exit code: 0 Mar 19 00:31:01.098: INFO: Unexpected error: <*errors.StatusError | 0xc0002bbe00>: error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] error dialing backend: dial tcp 10.138.0.5:10250: i/o timeout In [BeforeEach] at: test/e2e/storage/utils/local.go:201 @ 03/19/23 00:31:01.099 < Exit [BeforeEach] Pod with node different from PV's NodeAffinity - test/e2e/storage/persistent_volumes-local.go:365 @ 03/19/23 00:31:01.099 (13.681s) > Enter [AfterEach] Pod with node different from PV's NodeAffinity - test/e2e/storage/persistent_volumes-local.go:381 @ 03/19/23 00:31:01.099 STEP: Cleaning up PVC and PV - test/e2e/storage/persistent_volumes-local.go:863 @ 03/19/23 00:31:01.099 [PANICKED] Test Panicked In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260 @ 03/19/23 00:31:01.099 runtime error: invalid memory address or nil pointer dereference Full Stack Trace k8s.io/kubernetes/test/e2e/storage.cleanupLocalPVCsPVs({0x7fe6941a06d8, 0xc002054cc0}, 0xc002fd8870, {0xc003113f40, 0x1, 0xc003113f18?}) test/e2e/storage/persistent_volumes-local.go:864 +0xcd k8s.io/kubernetes/test/e2e/storage.cleanupLocalVolumes({0x7fe6941a06d8, 0xc002054cc0}, 0xc002fd8870, {0xc00320bf40, 0x1, 0x62d0740?}) test/e2e/storage/persistent_volumes-local.go:873 +0x36 k8s.io/kubernetes/test/e2e/storage.glob..func23.4.2({0x7fe6941a06d8, 0xc002054cc0}) test/e2e/storage/persistent_volumes-local.go:382 +0x51 < Exit [AfterEach] Pod with node different from PV's NodeAffinity - test/e2e/storage/persistent_volumes-local.go:381 @ 03/19/23 00:31:01.099 (0s) > Enter [AfterEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:01.099 Mar 19 00:31:01.099: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:01.18 (81ms) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:01.18 < Exit [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:01.18 (0s) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - dump namespaces | framework.go:209 @ 03/19/23 00:31:01.18 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:01.18 STEP: Collecting events from namespace "persistent-local-volumes-test-8935". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:01.18 STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:01.222 Mar 19 00:31:01.222: INFO: At 2023-03-19 00:30:47 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-shjgn: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-8935/hostexec-bootstrap-e2e-minion-group-tplt-shjgn to bootstrap-e2e-minion-group-tplt Mar 19 00:31:01.222: INFO: At 2023-03-19 00:30:47 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-shjgn: {kubelet bootstrap-e2e-minion-group-tplt} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:01.222: INFO: At 2023-03-19 00:30:53 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-shjgn: {kubelet bootstrap-e2e-minion-group-tplt} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 5.043014762s (5.043062224s including waiting) Mar 19 00:31:01.222: INFO: At 2023-03-19 00:30:53 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-shjgn: {kubelet bootstrap-e2e-minion-group-tplt} Created: Created container agnhost-container Mar 19 00:31:01.222: INFO: At 2023-03-19 00:30:53 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-tplt-shjgn: {kubelet bootstrap-e2e-minion-group-tplt} Started: Started container agnhost-container Mar 19 00:31:01.269: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:01.269: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn bootstrap-e2e-minion-group-tplt Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:47 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:54 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:54 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:47 +0000 UTC }] Mar 19 00:31:01.269: INFO: Mar 19 00:31:01.361: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:01.404: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:01.404: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:01.465: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:06.521: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:06.521: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.564: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:06.564: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.611: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.693: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:06.693: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:06.693: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:06.693: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:06.693: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:06.693: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:06.693: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:06.693: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:06.693: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:06.693: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:06.693: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:06.693: INFO: e2e-dns-utils started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:06.693: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:06.693: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:06.693: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:06.693: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:06.693: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:06.919: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:06.919: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:06.971: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 1511 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:06.971: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:07.047: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:07.311: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:07.311: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:07.311: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:07.311: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:07.311: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:07.311: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:07.311: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:07.311: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:07.311: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:07.311: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:07.311: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:07.311: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:07.311: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Init container init-volume-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:07.311: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:07.311: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:07.311: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:07.311: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:07.311: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:07.311: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:07.311: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:07.311: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:07.311: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:07.311: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:07.311: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:07.989: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:07.989: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:08.097: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:08.098: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:08.231: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:08.393: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:08.393: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:08.393: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:08.393: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:08.393: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.393: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:08.393: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:08.393: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:08.393: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:08.393: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.393: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-ljl6c started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.393: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:08.393: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:08.393: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:08.393: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:08.393: INFO: liveness-0d070cba-f6d9-4abd-977a-06769328147f started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:08.393: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:08.393: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container c ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-bfrv9 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.393: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-q5q7f started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.393: INFO: simpletest.rc-b7gtn started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.393: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:08.393: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:08.393: INFO: simpletest.rc-dbhb8 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:08.393: INFO: simpletest.rc-j99qw started at <nil> (0+0 container statuses recorded) Mar 19 00:31:12.598: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:12.598: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:12.706: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:12.706: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:12.911: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:18.138: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-zt76: error trying to reach service: dial tcp 10.138.0.3:10250: i/o timeout END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:18.139 (16.959s) < Exit [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - dump namespaces | framework.go:209 @ 03/19/23 00:31:18.139 (16.959s) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - tear down framework | framework.go:206 @ 03/19/23 00:31:18.139 STEP: Destroying namespace "persistent-local-volumes-test-8935" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:18.139 < Exit [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - tear down framework | framework.go:206 @ 03/19/23 00:31:18.183 (44ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:18.183 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:18.183 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sPersistentVolumes\-local\s\s\[Volume\stype\:\sblock\]\sOne\spod\srequesting\sone\sprebound\sPVC\sshould\sbe\sable\sto\smount\svolume\sand\sread\sfrom\spod1$'
[FAILED] error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout In [BeforeEach] at: test/e2e/storage/utils/local.go:135 @ 03/19/23 00:31:03.765 There were additional failures detected after the initial failure. These are visible in the timelinefrom junit_01.xml
> Enter [BeforeEach] [sig-storage] PersistentVolumes-local - set up framework | framework.go:191 @ 03/19/23 00:30:49.756 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:49.756 Mar 19 00:30:49.756: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:49.757 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:49.917 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:50.016 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - set up framework | framework.go:191 @ 03/19/23 00:30:50.151 (395ms) > Enter [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:50.151 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:50.151 (0s) > Enter [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/storage/persistent_volumes-local.go:161 @ 03/19/23 00:30:50.151 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/storage/persistent_volumes-local.go:161 @ 03/19/23 00:30:50.3 (149ms) > Enter [BeforeEach] [Volume type: block] - test/e2e/storage/persistent_volumes-local.go:198 @ 03/19/23 00:30:50.3 STEP: Initializing test volumes - test/e2e/storage/persistent_volumes-local.go:1117 @ 03/19/23 00:30:50.364 STEP: Creating block device on node "bootstrap-e2e-minion-group-g5rl" using path "/tmp/local-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9" - test/e2e/storage/utils/local.go:125 @ 03/19/23 00:30:50.364 Mar 19 00:30:58.628: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir -p /tmp/local-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9 && dd if=/dev/zero of=/tmp/local-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9/file] Namespace:persistent-local-volumes-test-3433 PodName:hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 19 00:30:58.628: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:30:58.629: INFO: ExecWithOptions: Clientset creation Mar 19 00:30:58.629: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/persistent-local-volumes-test-3433/pods/hostexec-bootstrap-e2e-minion-group-g5rl-55kh9/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+-p+%2Ftmp%2Flocal-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9+%26%26+dd+if%3D%2Fdev%2Fzero+of%3D%2Ftmp%2Flocal-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9%2Ffile+bs%3D4096+count%3D5120+%26%26+losetup+-f+%2Ftmp%2Flocal-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9%2Ffile&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:03.765: INFO: exec bootstrap-e2e-minion-group-g5rl: command: mkdir -p /tmp/local-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9 && dd if=/dev/zero of=/tmp/local-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9/file bs=4096 count=5120 && losetup -f /tmp/local-volume-test-a0961ec4-659c-4e47-b45e-279be701a4d9/file Mar 19 00:31:03.765: INFO: exec bootstrap-e2e-minion-group-g5rl: stdout: "" Mar 19 00:31:03.765: INFO: exec bootstrap-e2e-minion-group-g5rl: stderr: "" Mar 19 00:31:03.765: INFO: exec bootstrap-e2e-minion-group-g5rl: exit code: 0 Mar 19 00:31:03.765: INFO: Unexpected error: <*errors.StatusError | 0xc001dee6e0>: error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] error dialing backend: dial tcp 10.138.0.4:10250: i/o timeout In [BeforeEach] at: test/e2e/storage/utils/local.go:135 @ 03/19/23 00:31:03.765 < Exit [BeforeEach] [Volume type: block] - test/e2e/storage/persistent_volumes-local.go:198 @ 03/19/23 00:31:03.765 (13.465s) > Enter [AfterEach] [Volume type: block] - test/e2e/storage/persistent_volumes-local.go:211 @ 03/19/23 00:31:03.765 [FAILED] no test volume to cleanup In [AfterEach] at: test/e2e/storage/persistent_volumes-local.go:216 @ 03/19/23 00:31:03.765 < Exit [AfterEach] [Volume type: block] - test/e2e/storage/persistent_volumes-local.go:211 @ 03/19/23 00:31:03.765 (0s) > Enter [AfterEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:03.765 Mar 19 00:31:03.765: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:03.812 (46ms) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:03.812 < Exit [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:03.812 (0s) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - dump namespaces | framework.go:209 @ 03/19/23 00:31:03.812 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:03.812 STEP: Collecting events from namespace "persistent-local-volumes-test-3433". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:03.812 STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:03.853 Mar 19 00:31:03.853: INFO: At 2023-03-19 00:30:50 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-55kh9: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-3433/hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 to bootstrap-e2e-minion-group-g5rl Mar 19 00:31:03.853: INFO: At 2023-03-19 00:30:51 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-55kh9: {kubelet bootstrap-e2e-minion-group-g5rl} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:03.853: INFO: At 2023-03-19 00:30:55 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-55kh9: {kubelet bootstrap-e2e-minion-group-g5rl} Pulled: Successfully pulled image "registry.k8s.io/e2e-test-images/agnhost:2.43" in 4.637309329s (4.637334098s including waiting) Mar 19 00:31:03.853: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-55kh9: {kubelet bootstrap-e2e-minion-group-g5rl} Created: Created container agnhost-container Mar 19 00:31:03.853: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-g5rl-55kh9: {kubelet bootstrap-e2e-minion-group-g5rl} Started: Started container agnhost-container Mar 19 00:31:03.895: INFO: POD NODE PHASE GRACE CONDITIONS Mar 19 00:31:03.895: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 bootstrap-e2e-minion-group-g5rl Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:50 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:56 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:56 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-19 00:30:50 +0000 UTC }] Mar 19 00:31:03.895: INFO: Mar 19 00:31:03.990: INFO: Logging node info for node bootstrap-e2e-master Mar 19 00:31:04.032: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 974d9aab-f47b-455e-9738-8ace2169f1a4 648 0 2023-03-19 00:28:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2023-03-19 00:29:12 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:12 +0000 UTC,LastTransitionTime:2023-03-19 00:29:12 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:15 +0000 UTC,LastTransitionTime:2023-03-19 00:28:58 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.145.101.188,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b650e6ef2be2a5d664f1a05cbc20f466,SystemUUID:b650e6ef-2be2-a5d6-64f1-a05cbc20f466,BootID:a25127a3-d22d-4942-bea3-bb47808e3a42,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:121910627,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:113796605,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:59680240,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:04.032: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 19 00:31:04.074: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 19 00:31:09.145: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: dial tcp 10.138.0.2:10250: i/o timeout Mar 19 00:31:09.145: INFO: Logging node info for node bootstrap-e2e-minion-group-494g Mar 19 00:31:09.229: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-494g 24c1f46b-9b91-4941-ad24-b980522ac7e4 1378 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-494g kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:30:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-494g,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:30:55 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.6,},NodeAddress{Type:ExternalIP,Address:35.203.176.120,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-494g.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:40cff702419f976d4b760cec6a332943,SystemUUID:40cff702-419f-976d-4b76-0cec6a332943,BootID:83acb8cc-3cd1-4e7f-8ac6-d8835d3cb158,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:09.229: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-494g Mar 19 00:31:09.324: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-494g Mar 19 00:31:09.565: INFO: metadata-proxy-v0.1-42p22 started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:09.565: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:09.565: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:09.565: INFO: pod-subpath-test-inlinevolume-h5q6 started at 2023-03-19 00:30:57 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Init container init-volume-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:09.565: INFO: Container test-container-subpath-inlinevolume-h5q6 ready: true, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-6gswg started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.565: INFO: simpletest.rc-slljn started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.565: INFO: konnectivity-agent-c56nw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:09.565: INFO: pod-subpath-test-configmap-8sm7 started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container test-container-subpath-configmap-8sm7 ready: true, restart count 0 Mar 19 00:31:09.565: INFO: e2e-net-server started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container e2e-net-server ready: true, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-ztqkh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-5vxl5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-bgwp2 started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.565: INFO: simpletest.rc-pnjjn started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.565: INFO: execpod-noendpointstrx5g started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:09.565: INFO: hostexec-bootstrap-e2e-minion-group-494g-jd2vs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container agnhost-container ready: false, restart count 0 Mar 19 00:31:09.565: INFO: test-rollover-deployment-57777854c9-ptzxc started at 2023-03-19 00:31:02 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container agnhost ready: true, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-l8ssf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-rpvgh started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-wzssg started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-jdnxl started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.565: INFO: simpletest.rc-pd2gt started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-8vdgj started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.565: INFO: simpletest.rc-wpfgd started at <nil> (0+0 container statuses recorded) Mar 19 00:31:09.565: INFO: simpletest.rc-4z9x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-gj7hb started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: hostexec-bootstrap-e2e-minion-group-494g-k85td started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-4dwp5 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-nbwvl started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:09.565: INFO: kube-proxy-bootstrap-e2e-minion-group-494g started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:09.565: INFO: coredns-8f5847b64-w27fp started at 2023-03-19 00:29:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:09.565: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:09.565: INFO: test-rollover-controller-mfm7l started at 2023-03-19 00:30:55 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container httpd ready: true, restart count 0 Mar 19 00:31:09.565: INFO: simpletest.rc-7vn45 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:09.565: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.166: INFO: Latency metrics for node bootstrap-e2e-minion-group-494g Mar 19 00:31:11.166: INFO: Logging node info for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:11.210: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-g5rl 083bceec-d07b-4382-ad25-ff4e3c81aa51 1511 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-g5rl kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-g5rl topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-6864":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-read-write-once-pod-5648":"bootstrap-e2e-minion-group-g5rl","csi-hostpath-volume-expand-62":"bootstrap-e2e-minion-group-g5rl"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:31:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-g5rl,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:54 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.233.240.248,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-g5rl.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:14310f0b694495b049463d7d5dc3fa14,SystemUUID:14310f0b-6944-95b0-4946-3d7d5dc3fa14,BootID:049da10d-12b5-43f7-9b2f-3a67d8db485e,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:11.211: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:11.261: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:11.517: INFO: pod-csi-inline-volumes started at 2023-03-19 00:30:49 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container pod-csi-inline-volumes ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-7sxn7 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-rsmrn started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-xdf8g started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-5jdgw started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: pod-7d5a3317-e722-4fa7-8cbf-8ed56da69143 started at 2023-03-19 00:31:11 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:11.517: INFO: kube-proxy-bootstrap-e2e-minion-group-g5rl started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:11.517: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:58 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:11.517: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:11.517: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:11.517: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:11.517: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-xg9sm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-jxspp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: metadata-proxy-v0.1-m92jj started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:11.517: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:11.517: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:11.517: INFO: nfs-server started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nfs-server ready: true, restart count 0 Mar 19 00:31:11.517: INFO: hostexec-bootstrap-e2e-minion-group-g5rl-55kh9 started at 2023-03-19 00:30:50 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-6m7f4 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-jzxp6 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-b987l started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: simpletest.rc-4l96m started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.517: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.517: INFO: volume-snapshot-controller-0 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 19 00:31:11.518: INFO: l7-default-backend-856d874f49-bkgvd started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container default-http-backend ready: true, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-jg8xq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-c42st started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-6q8kv started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-69cfd started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: pfpod started at 2023-03-19 00:30:48 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:11.518: INFO: Container portforwardtester ready: true, restart count 0 Mar 19 00:31:11.518: INFO: Container readiness ready: true, restart count 0 Mar 19 00:31:11.518: INFO: pod-subpath-test-inlinevolume-km5v started at 2023-03-19 00:31:05 +0000 UTC (1+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Init container init-volume-inlinevolume-km5v ready: true, restart count 0 Mar 19 00:31:11.518: INFO: Container test-container-subpath-inlinevolume-km5v ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-2rgjn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-8vktm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-jw7ss started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: kube-dns-autoscaler-7b444c59c9-bpscw started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container autoscaler ready: true, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-2p9rs started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-kktj7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-qnn4q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-rt8qj started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-szxkk started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-w8jmk started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-frh8z started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: pod-subpath-test-dynamicpv-w4tj started at 2023-03-19 00:31:10 +0000 UTC (2+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Init container init-volume-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Init container test-init-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Container test-container-subpath-dynamicpv-w4tj ready: false, restart count 0 Mar 19 00:31:11.518: INFO: coredns-8f5847b64-f2nxf started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container coredns ready: true, restart count 0 Mar 19 00:31:11.518: INFO: konnectivity-agent-2blw2 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:11.518: INFO: csi-hostpathplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+7 container statuses recorded) Mar 19 00:31:11.518: INFO: Container csi-attacher ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Container csi-provisioner ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Container csi-resizer ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Container csi-snapshotter ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Container hostpath ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Container liveness-probe ready: false, restart count 0 Mar 19 00:31:11.518: INFO: Container node-driver-registrar ready: false, restart count 0 Mar 19 00:31:11.518: INFO: simpletest.rc-6x66r started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:11.518: INFO: pod-55f6ff00-b7bd-449e-a96e-935dccef36ac started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:11.518: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:18.771: INFO: Latency metrics for node bootstrap-e2e-minion-group-g5rl Mar 19 00:31:18.771: INFO: Logging node info for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:18.839: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-tplt 296f4ed6-83ea-485c-88af-690fa7131271 1786 0 2023-03-19 00:28:53 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-tplt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-tplt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-workload-572":"bootstrap-e2e-minion-group-tplt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-19 00:28:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-19 00:28:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2023-03-19 00:28:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:31:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2023-03-19 00:31:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-tplt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:58 +0000 UTC,LastTransitionTime:2023-03-19 00:28:57 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:53 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:31:07 +0000 UTC,LastTransitionTime:2023-03-19 00:28:54 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.203.149.116,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-tplt.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:115fb61059e5596c0ee09af3b265a263,SystemUUID:115fb610-59e5-596c-0ee0-9af3b265a263,BootID:12d875d4-f571-406c-8bbe-5db27453f420,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-workload-572^53315ad6-c5ed-11ed-abd3-ea406d295584,DevicePath:,},},Config:nil,},} Mar 19 00:31:18.839: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:18.921: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-tplt Mar 19 00:31:19.097: INFO: metadata-proxy-v0.1-wffvm started at 2023-03-19 00:28:54 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.097: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:19.097: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:19.097: INFO: konnectivity-agent-gdrg6 started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:19.097: INFO: hostexec-bootstrap-e2e-minion-group-tplt-shjgn started at 2023-03-19 00:30:47 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:19.097: INFO: probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container probe-test-05b2bea6-5ecc-452c-9f0b-dbb5192233e6 ready: false, restart count 0 Mar 19 00:31:19.097: INFO: pvc-volume-tester-rbvls started at 2023-03-19 00:31:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container volume-tester ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-9hdwf started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-dh86v started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-c5xmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: successful-jobs-history-limit-27986431-558n8 started at 2023-03-19 00:31:00 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container c ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-fqq8n started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-p6lgg started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-bmrz6 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: httpd-deployment-55bcbdf6f4-nzqjk started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-j99qw started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-4fqhh started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: csi-mockplugin-0 started at 2023-03-19 00:30:51 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:19.097: INFO: Container csi-provisioner ready: true, restart count 0 Mar 19 00:31:19.097: INFO: Container driver-registrar ready: true, restart count 0 Mar 19 00:31:19.097: INFO: Container mock ready: true, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-rvz2x started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-rtgln started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-bzq2b started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-fpdbm started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-ljl6c started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-qxrn8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-jhgv8 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-kltzl started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-8pbtv started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: kube-proxy-bootstrap-e2e-minion-group-tplt started at 2023-03-19 00:28:53 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:19.097: INFO: metrics-server-v0.5.2-57d47cbf5-tbfsg started at 2023-03-19 00:29:18 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:19.097: INFO: Container metrics-server ready: true, restart count 0 Mar 19 00:31:19.097: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 19 00:31:19.097: INFO: csi-mockplugin-attacher-0 started at 2023-03-19 00:30:51 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container csi-attacher ready: true, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-bfrv9 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:19.097: INFO: pvc-tester-lmc2x started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-fphmv started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-zvjzr started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-9ss5w started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: pod-client started at 2023-03-19 00:31:13 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container pod-client ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-q5q7f started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-b7gtn started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-k4vlf started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:19.097: INFO: simpletest.rc-dbhb8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:19.097: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.320: INFO: Latency metrics for node bootstrap-e2e-minion-group-tplt Mar 19 00:31:20.320: INFO: Logging node info for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.363: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-zt76 69f133c2-4f25-4553-a065-fda792c24378 674 0 2023-03-19 00:28:52 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-zt76 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2023-03-19 00:28:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-19 00:28:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2023-03-19 00:29:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-19 00:29:23 +0000 UTC FieldsV1 {"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-infra-e2e-boskos-083/us-west1-b/bootstrap-e2e-minion-group-zt76,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-19 00:28:57 +0000 UTC,LastTransitionTime:2023-03-19 00:28:56 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-19 00:29:02 +0000 UTC,LastTransitionTime:2023-03-19 00:29:02 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-19 00:29:23 +0000 UTC,LastTransitionTime:2023-03-19 00:28:52 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.80.78,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-zt76.c.k8s-infra-e2e-boskos-083.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:28b6a4322652d0a5cd60a49c68455a28,SystemUUID:28b6a432-2652-d0a5-cd60-a49c68455a28,BootID:d8c7f8c4-899e-4183-acd1-33b5a02443d7,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-beta.0.22+fe91bc257b505e,KubeProxyVersion:v1.27.0-beta.0.22+fe91bc257b505e,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-beta.0.22_fe91bc257b505e],SizeBytes:72681314,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 19 00:31:20.363: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.433: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-zt76 Mar 19 00:31:20.592: INFO: pvc-tester-n6v47 started at 2023-03-19 00:31:05 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container write-pod ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-928rb started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-r77f8 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: external-provisioner-kh5pp started at 2023-03-19 00:30:46 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nfs-provisioner ready: true, restart count 0 Mar 19 00:31:20.592: INFO: e2e-net-exec started at 2023-03-19 00:30:58 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 19 00:31:20.592: INFO: kube-proxy-bootstrap-e2e-minion-group-zt76 started at 2023-03-19 00:28:52 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container kube-proxy ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-nb5tk started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-9kxh7 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-nrb9d started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-mlpmn started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: konnectivity-agent-jl7mj started at 2023-03-19 00:29:03 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 19 00:31:20.592: INFO: httpd-deployment-5cd84d4f9-pngsc started at 2023-03-19 00:31:18 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-xdvlv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-vzz8p started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-r4rx7 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: implicit-nonroot-uid started at 2023-03-19 00:31:19 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container implicit-nonroot-uid ready: false, restart count 0 Mar 19 00:31:20.592: INFO: hostexec-bootstrap-e2e-minion-group-zt76-6tdlz started at 2023-03-19 00:30:48 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container agnhost-container ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-pk55q started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-zfk5k started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-sck2h started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-cm7k8 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: dns-test-cff8fe3e-c2c5-48b0-8dc1-48e6b5438fcc started at 2023-03-19 00:31:18 +0000 UTC (0+3 container statuses recorded) Mar 19 00:31:20.592: INFO: Container jessie-querier ready: false, restart count 0 Mar 19 00:31:20.592: INFO: Container querier ready: false, restart count 0 Mar 19 00:31:20.592: INFO: Container webserver ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-4m46r started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-j9xqq started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-nlqsp started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-lss94 started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-bpdmm started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-m2wd9 started at 2023-03-19 00:31:10 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-s7q9z started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: pod-de4e88ee-387a-42e0-a9e4-8a03876dc49f started at 2023-03-19 00:30:57 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container write-pod ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-qz8x8 started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-vpkhp started at 2023-03-19 00:31:09 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: httpd-deployment-55bcbdf6f4-z75nw started at 2023-03-19 00:31:17 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container httpd ready: false, restart count 0 Mar 19 00:31:20.592: INFO: metadata-proxy-v0.1-xn95s started at 2023-03-19 00:28:53 +0000 UTC (0+2 container statuses recorded) Mar 19 00:31:20.592: INFO: Container metadata-proxy ready: true, restart count 0 Mar 19 00:31:20.592: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 19 00:31:20.592: INFO: e2e-net-client started at 2023-03-19 00:31:04 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container e2e-net-client ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-v2h4g started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: true, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-qt9cv started at 2023-03-19 00:31:07 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:20.592: INFO: simpletest.rc-k4vh7 started at 2023-03-19 00:31:08 +0000 UTC (0+1 container statuses recorded) Mar 19 00:31:20.592: INFO: Container nginx ready: false, restart count 0 Mar 19 00:31:25.097: INFO: Latency metrics for node bootstrap-e2e-minion-group-zt76 END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:25.097 (21.285s) < Exit [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - dump namespaces | framework.go:209 @ 03/19/23 00:31:25.097 (21.285s) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - tear down framework | framework.go:206 @ 03/19/23 00:31:25.097 STEP: Destroying namespace "persistent-local-volumes-test-3433" for this suite. - test/e2e/framework/framework.go:351 @ 03/19/23 00:31:25.097 < Exit [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - tear down framework | framework.go:206 @ 03/19/23 00:31:25.205 (108ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.205 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/19/23 00:31:25.205 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-storage\]\sPersistentVolumes\-local\s\s\[Volume\stype\:\sdir\-link\-bindmounted\]\sOne\spod\srequesting\sone\sprebound\sPVC\sshould\sbe\sable\sto\smount\svolume\sand\sread\sfrom\spod1$'
[FAILED] error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout In [BeforeEach] at: test/e2e/storage/utils/local.go:260 @ 03/19/23 00:31:04.473 There were additional failures detected after the initial failure. These are visible in the timeline
> Enter [BeforeEach] [sig-storage] PersistentVolumes-local - set up framework | framework.go:191 @ 03/19/23 00:30:48.125 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/19/23 00:30:48.125 Mar 19 00:30:48.125: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename persistent-local-volumes-test - test/e2e/framework/framework.go:250 @ 03/19/23 00:30:48.127 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/19/23 00:30:48.417 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/19/23 00:30:48.617 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - set up framework | framework.go:191 @ 03/19/23 00:30:48.761 (636ms) > Enter [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.761 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:33 @ 03/19/23 00:30:48.761 (0s) > Enter [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/storage/persistent_volumes-local.go:161 @ 03/19/23 00:30:48.761 < Exit [BeforeEach] [sig-storage] PersistentVolumes-local - test/e2e/storage/persistent_volumes-local.go:161 @ 03/19/23 00:30:48.858 (97ms) > Enter [BeforeEach] [Volume type: dir-link-bindmounted] - test/e2e/storage/persistent_volumes-local.go:198 @ 03/19/23 00:30:48.858 STEP: Initializing test volumes - test/e2e/storage/persistent_volumes-local.go:1117 @ 03/19/23 00:30:48.937 Mar 19 00:30:59.320: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c mkdir /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend && mount --bind /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend && ln -s /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69] Namespace:persistent-local-volumes-test-1547 PodName:hostexec-bootstrap-e2e-minion-group-494g-k85td ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 19 00:30:59.320: INFO: >>> kubeConfig: /workspace/.kube/config Mar 19 00:30:59.322: INFO: ExecWithOptions: Clientset creation Mar 19 00:30:59.322: INFO: ExecWithOptions: execute(POST https://34.145.101.188/api/v1/namespaces/persistent-local-volumes-test-1547/pods/hostexec-bootstrap-e2e-minion-group-494g-k85td/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=mkdir+%2Ftmp%2Flocal-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend+%26%26+mount+--bind+%2Ftmp%2Flocal-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend+%2Ftmp%2Flocal-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend+%26%26+ln+-s+%2Ftmp%2Flocal-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend+%2Ftmp%2Flocal-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) Mar 19 00:31:04.473: INFO: exec bootstrap-e2e-minion-group-494g: command: mkdir /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend && mount --bind /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend && ln -s /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69-backend /tmp/local-volume-test-c059db79-cdc3-4c70-b5e3-302e44e9bd69 Mar 19 00:31:04.473: INFO: exec bootstrap-e2e-minion-group-494g: stdout: "" Mar 19 00:31:04.473: INFO: exec bootstrap-e2e-minion-group-494g: stderr: "" Mar 19 00:31:04.473: INFO: exec bootstrap-e2e-minion-group-494g: exit code: 0 Mar 19 00:31:04.473: INFO: Unexpected error: <*errors.StatusError | 0xc000cc88c0>: error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout { ErrStatus: code: 500 message: 'error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout' metadata: {} status: Failure, } [FAILED] error dialing backend: dial tcp 10.138.0.6:10250: i/o timeout In [BeforeEach] at: test/e2e/storage/utils/local.go:260 @ 03/19/23 00:31:04.473 < Exit [BeforeEach] [Volume type: dir-link-bindmounted] - test/e2e/storage/persistent_volumes-local.go:198 @ 03/19/23 00:31:04.474 (15.616s) > Enter [AfterEach] [Volume type: dir-link-bindmounted] - test/e2e/storage/persistent_volumes-local.go:211 @ 03/19/23 00:31:04.474 [FAILED] no test volume to cleanup In [AfterEach] at: test/e2e/storage/persistent_volumes-local.go:216 @ 03/19/23 00:31:04.474 < Exit [AfterEach] [Volume type: dir-link-bindmounted] - test/e2e/storage/persistent_volumes-local.go:211 @ 03/19/23 00:31:04.474 (0s) > Enter [AfterEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:04.474 Mar 19 00:31:04.474: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-storage] PersistentVolumes-local - test/e2e/framework/node/init/init.go:33 @ 03/19/23 00:31:04.534 (60ms) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:04.534 < Exit [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - test/e2e/framework/metrics/init/init.go:35 @ 03/19/23 00:31:04.534 (0s) > Enter [DeferCleanup (Each)] [sig-storage] PersistentVolumes-local - dump namespaces | framework.go:209 @ 03/19/23 00:31:04.534 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/19/23 00:31:04.534 STEP: Collecting events from namespace "persistent-local-volumes-test-1547". - test/e2e/framework/debug/dump.go:42 @ 03/19/23 00:31:04.534 STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/19/23 00:31:04.581 Mar 19 00:31:04.581: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-k85td: {default-scheduler } Scheduled: Successfully assigned persistent-local-volumes-test-1547/hostexec-bootstrap-e2e-minion-group-494g-k85td to bootstrap-e2e-minion-group-494g Mar 19 00:31:04.581: INFO: At 2023-03-19 00:30:49 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-k85td: {kubelet bootstrap-e2e-minion-group-494g} Pulling: Pulling image "registry.k8s.io/e2e-test-images/agnhost:2.43" Mar 19 00:31:04.581: INFO: At 2023-03-19 00:30:56 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-494g-k85td: {kubelet bootstrap-e2e-minion-group-494g} Pulled: Successfully pulled image