go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\smutate\scustom\sresource\swith\sdifferent\sstored\sversion\s\[Conformance\]$'
[FAILED] failed to create custom resource cr-instance-1 in namespace: webhook-2754: Internal error occurred: failed calling webhook "mutate-custom-resource-data-stage-1.k8s.io": failed to call webhook: Post "https://e2e-test-webhook.webhook-2754.svc:8443/mutating-custom-resource?timeout=10s": context deadline exceeded In [It] at: test/e2e/apimachinery/webhook.go:1896 @ 03/16/23 14:52:11.427from junit_01.xml
> Enter [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - set up framework | framework.go:191 @ 03/16/23 14:51:54.371 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:51:54.371 Mar 16 14:51:54.371: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename webhook - test/e2e/framework/framework.go:250 @ 03/16/23 14:51:54.373 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:51:54.497 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:51:54.578 < Exit [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - set up framework | framework.go:191 @ 03/16/23 14:51:54.66 (289ms) > Enter [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:51:54.66 < Exit [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:51:54.66 (0s) > Enter [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:91 @ 03/16/23 14:51:54.66 STEP: Setting up server cert - test/e2e/apimachinery/webhook.go:99 @ 03/16/23 14:51:54.832 STEP: Create role binding to let webhook read extension-apiserver-authentication - test/e2e/apimachinery/webhook.go:709 @ 03/16/23 14:51:55.065 STEP: Deploying the webhook pod - test/e2e/apimachinery/webhook.go:741 @ 03/16/23 14:51:55.113 STEP: Wait for the deployment to be ready - test/e2e/apimachinery/webhook.go:811 @ 03/16/23 14:51:55.209 Mar 16 14:51:55.383: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2023, time.March, 16, 14, 51, 55, 0, time.Local), LastTransitionTime:time.Date(2023, time.March, 16, 14, 51, 55, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2023, time.March, 16, 14, 51, 55, 0, time.Local), LastTransitionTime:time.Date(2023, time.March, 16, 14, 51, 55, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-7497495989\" is progressing."}}, CollisionCount:(*int32)(nil)} STEP: Deploying the webhook service - test/e2e/apimachinery/webhook.go:817 @ 03/16/23 14:51:57.436 STEP: Verifying the service has paired with the endpoint - test/e2e/apimachinery/webhook.go:840 @ 03/16/23 14:51:57.5 Mar 16 14:51:58.500: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1 < Exit [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:91 @ 03/16/23 14:51:58.548 (3.888s) > Enter [It] should mutate custom resource with different stored version [Conformance] - test/e2e/apimachinery/webhook.go:314 @ 03/16/23 14:51:58.548 Mar 16 14:51:58.548: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Registering the mutating webhook for custom resource e2e-test-webhook-6798-crds.webhook.example.com via the AdmissionRegistration API - test/e2e/apimachinery/webhook.go:1690 @ 03/16/23 14:51:59.239 STEP: Creating a custom resource while v1 is storage version - test/e2e/apimachinery/webhook.go:1880 @ 03/16/23 14:51:59.382 Mar 16 14:52:11.427: INFO: Unexpected error: failed to create custom resource cr-instance-1 in namespace: webhook-2754: <*errors.StatusError | 0xc0040ee640>: Internal error occurred: failed calling webhook "mutate-custom-resource-data-stage-1.k8s.io": failed to call webhook: Post "https://e2e-test-webhook.webhook-2754.svc:8443/mutating-custom-resource?timeout=10s": context deadline exceeded { ErrStatus: apiVersion: v1 code: 500 details: causes: - message: 'failed calling webhook "mutate-custom-resource-data-stage-1.k8s.io": failed to call webhook: Post "https://e2e-test-webhook.webhook-2754.svc:8443/mutating-custom-resource?timeout=10s": context deadline exceeded' kind: Status message: 'Internal error occurred: failed calling webhook "mutate-custom-resource-data-stage-1.k8s.io": failed to call webhook: Post "https://e2e-test-webhook.webhook-2754.svc:8443/mutating-custom-resource?timeout=10s": context deadline exceeded' metadata: {} reason: InternalError status: Failure, } [FAILED] failed to create custom resource cr-instance-1 in namespace: webhook-2754: Internal error occurred: failed calling webhook "mutate-custom-resource-data-stage-1.k8s.io": failed to call webhook: Post "https://e2e-test-webhook.webhook-2754.svc:8443/mutating-custom-resource?timeout=10s": context deadline exceeded In [It] at: test/e2e/apimachinery/webhook.go:1896 @ 03/16/23 14:52:11.427 < Exit [It] should mutate custom resource with different stored version [Conformance] - test/e2e/apimachinery/webhook.go:314 @ 03/16/23 14:52:11.427 (12.879s) > Enter [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:52:11.427 Mar 16 14:52:11.427: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:52:11.478 (51ms) > Enter [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:106 @ 03/16/23 14:52:11.478 < Exit [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:106 @ 03/16/23 14:52:11.748 (270ms) > Enter [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:1761 @ 03/16/23 14:52:11.748 < Exit [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:1761 @ 03/16/23 14:52:11.802 (54ms) > Enter [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:319 @ 03/16/23 14:52:11.802 < Exit [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/apimachinery/webhook.go:319 @ 03/16/23 14:52:12.521 (719ms) > Enter [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:52:12.521 < Exit [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:52:12.521 (0s) > Enter [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - dump namespaces | framework.go:209 @ 03/16/23 14:52:12.521 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:52:12.521 STEP: Collecting events from namespace "webhook-2754". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:52:12.521 STEP: Found 8 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:52:12.609 Mar 16 14:52:12.609: INFO: At 2023-03-16 14:51:55 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-7497495989 to 1 Mar 16 14:52:12.609: INFO: At 2023-03-16 14:51:55 +0000 UTC - event for sample-webhook-deployment-7497495989: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-7497495989-q5md5 Mar 16 14:52:12.609: INFO: At 2023-03-16 14:51:55 +0000 UTC - event for sample-webhook-deployment-7497495989-q5md5: {default-scheduler } Scheduled: Successfully assigned webhook-2754/sample-webhook-deployment-7497495989-q5md5 to bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:12.609: INFO: At 2023-03-16 14:51:56 +0000 UTC - event for sample-webhook-deployment-7497495989-q5md5: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:52:12.609: INFO: At 2023-03-16 14:51:56 +0000 UTC - event for sample-webhook-deployment-7497495989-q5md5: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container sample-webhook Mar 16 14:52:12.609: INFO: At 2023-03-16 14:51:56 +0000 UTC - event for sample-webhook-deployment-7497495989-q5md5: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container sample-webhook Mar 16 14:52:12.609: INFO: At 2023-03-16 14:52:11 +0000 UTC - event for sample-webhook-deployment-7497495989-q5md5: {kubelet bootstrap-e2e-minion-group-hzx6} Killing: Stopping container sample-webhook Mar 16 14:52:12.609: INFO: At 2023-03-16 14:52:11 +0000 UTC - event for sample-webhook-deployment-7497495989-q5md5: {kubelet bootstrap-e2e-minion-group-hzx6} Unhealthy: Readiness probe failed: Get "https://10.64.2.180:8444/readyz": dial tcp 10.64.2.180:8444: connect: connection refused Mar 16 14:52:12.697: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:52:12.697: INFO: Mar 16 14:52:12.770: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:52:13.005: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 660 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:47:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:13.006: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:52:13.279: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:52:13.451: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:52:13.451: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:52:13.451: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:52:13.451: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:52:13.451: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:52:13.451: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:52:13.451: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:13.451: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:13.451: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:13.451: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:52:13.451: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.451: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:52:13.746: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:52:13.746: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:13.794: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 8265 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:13.795: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:13.848: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:13.945: INFO: dns-test-f0228d29-e7d2-4382-b0cf-298487474d01 started at 2023-03-16 14:51:01 +0000 UTC (0+3 container statuses recorded) Mar 16 14:52:13.945: INFO: Container jessie-querier ready: false, restart count 1 Mar 16 14:52:13.945: INFO: Container querier ready: false, restart count 1 Mar 16 14:52:13.945: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:13.945: INFO: simpletest-rc-to-be-deleted-h66mg started at 2023-03-16 14:51:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:13.945: INFO: pod-should-be-evicted4f13af73-bec5-44f3-887b-797d587a0ad5 started at 2023-03-16 14:51:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container bar ready: true, restart count 0 Mar 16 14:52:13.945: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:52:13.945: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container httpd ready: true, restart count 0 Mar 16 14:52:13.945: INFO: busybox-a5a35a8e-81ce-4aed-8ded-436003287c63 started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container busybox ready: true, restart count 0 Mar 16 14:52:13.945: INFO: test-rolling-update-with-lb-69fcddc54f-6qbjs started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:13.945: INFO: simpletest-rc-to-be-deleted-fwrxw started at 2023-03-16 14:51:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:13.945: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:13.945: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:52:13.945: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:13.945: INFO: netserver-0 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:13.945: INFO: netserver-0 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:13.945: INFO: simpletest-rc-to-be-deleted-jrqhv started at 2023-03-16 14:51:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:13.945: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:13.945: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:13.945: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:13.945: INFO: simpletest-rc-to-be-deleted-b9j9s started at 2023-03-16 14:51:19 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:13.945: INFO: simpletest-rc-to-be-deleted-gslfn started at 2023-03-16 14:51:19 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:13.945: INFO: simpletest-rc-to-be-deleted-hgzkd started at 2023-03-16 14:51:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:13.945: INFO: test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:13.945: INFO: simpletest-rc-to-be-deleted-9bjl5 started at 2023-03-16 14:51:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:13.945: INFO: dns-test-72c4b60b-47b2-4966-b176-d324ed05e48b started at 2023-03-16 14:51:35 +0000 UTC (0+3 container statuses recorded) Mar 16 14:52:13.945: INFO: Container jessie-querier ready: true, restart count 1 Mar 16 14:52:13.945: INFO: Container querier ready: true, restart count 1 Mar 16 14:52:13.945: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:13.945: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:13.945: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:52:13.945: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:52:13.945: INFO: host-test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:13.945: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:13.945: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:13.945: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:13.945: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:13.945: INFO: test-host-network-pod started at 2023-03-16 14:52:07 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:13.945: INFO: Container busybox-1 ready: true, restart count 0 Mar 16 14:52:13.945: INFO: Container busybox-2 ready: true, restart count 0 Mar 16 14:52:13.945: INFO: pod-handle-http-request started at 2023-03-16 14:51:36 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:13.945: INFO: Container container-handle-8090-request ready: true, restart count 0 Mar 16 14:52:13.945: INFO: Container container-handle-9090-request ready: true, restart count 0 Mar 16 14:52:14.241: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:14.241: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:14.285: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:14.285: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:14.332: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:14.423: INFO: probe-test-b81d4fc4-f22a-4b94-80a5-af7def1bb6a9 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container probe-test-b81d4fc4-f22a-4b94-80a5-af7def1bb6a9 ready: true, restart count 0 Mar 16 14:52:14.423: INFO: pause-pod-0 started at 2023-03-16 14:51:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:14.423: INFO: pod2 started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container container1 ready: false, restart count 0 Mar 16 14:52:14.423: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:14.423: INFO: execpodnsv2z started at 2023-03-16 14:52:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:14.423: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:14.423: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container coredns ready: true, restart count 0 Mar 16 14:52:14.423: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:14.423: INFO: verify-service-down-host-exec-pod started at 2023-03-16 14:51:42 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:14.423: INFO: simpletest-rc-to-be-deleted-dh2ln started at 2023-03-16 14:51:19 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:14.423: INFO: pod1 started at 2023-03-16 14:52:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container container1 ready: false, restart count 0 Mar 16 14:52:14.423: INFO: pod1 started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:14.423: INFO: pod1 started at 2023-03-16 14:52:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:14.423: INFO: concurrent-27982972-6527z started at 2023-03-16 14:52:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container c ready: true, restart count 0 Mar 16 14:52:14.423: INFO: test-rolling-update-with-lb-69fcddc54f-chjs7 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:14.423: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:14.423: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:14.423: INFO: netserver-1 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:14.423: INFO: ss-0 started at 2023-03-16 14:52:03 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:14.423: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:14.423: INFO: simpletest-rc-to-be-deleted-fndt2 started at 2023-03-16 14:51:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:14.423: INFO: externalname-service-6q82d started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:52:14.423: INFO: simpletest-rc-to-be-deleted-cljr5 started at 2023-03-16 14:51:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container nginx ready: true, restart count 0 Mar 16 14:52:14.423: INFO: busybox-scheduling-b42fe286-a3b5-4398-a9c1-a48f5ac2756d started at 2023-03-16 14:51:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container busybox-scheduling-b42fe286-a3b5-4398-a9c1-a48f5ac2756d ready: true, restart count 0 Mar 16 14:52:14.423: INFO: execpodf4mm6 started at 2023-03-16 14:51:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:14.423: INFO: successful-jobs-history-limit-27982972-5xjm6 started at 2023-03-16 14:52:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container c ready: false, restart count 0 Mar 16 14:52:14.423: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:14.423: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:14.423: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:14.423: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:14.423: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:15.440: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:15.440: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:15.494: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 6652 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:15.495: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:15.549: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:52:15.629: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:15.629: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:52:15.629: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container coredns ready: true, restart count 0 Mar 16 14:52:15.629: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:15.629: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:15.629: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:15.629: INFO: netserver-2 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:15.629: INFO: dns-test-29bf3d35-0e3a-49a8-a83c-0276b869e7de started at 2023-03-16 14:51:57 +0000 UTC (0+3 container statuses recorded) Mar 16 14:52:15.629: INFO: Container jessie-querier ready: true, restart count 0 Mar 16 14:52:15.629: INFO: Container querier ready: true, restart count 0 Mar 16 14:52:15.629: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:15.629: INFO: execpodpplld started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:15.629: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:15.629: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:15.629: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:15.629: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:15.629: INFO: run-test-2 started at 2023-03-16 14:52:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container run-test-2 ready: false, restart count 0 Mar 16 14:52:15.629: INFO: externalname-service-22tpf started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:52:15.629: INFO: sample-webhook-deployment-7497495989-tmzfq started at 2023-03-16 14:52:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container sample-webhook ready: true, restart count 0 Mar 16 14:52:15.629: INFO: kube-dns-autoscaler-7b444c59c9-2q9j6 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:52:15.629: INFO: httpd started at 2023-03-16 14:51:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container httpd ready: true, restart count 0 Mar 16 14:52:15.629: INFO: replace-27982971-kn4jb started at 2023-03-16 14:51:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container c ready: true, restart count 0 Mar 16 14:52:15.629: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:15.629: INFO: concurrent-27982971-2w2br started at 2023-03-16 14:51:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:15.629: INFO: Container c ready: true, restart count 0 Mar 16 14:52:15.909: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:15.909: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:15.962: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 10041 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:52:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:15.963: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:16.010: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:16.133: INFO: privileged-pod started at 2023-03-16 14:51:57 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:16.133: INFO: Container not-privileged-container ready: true, restart count 0 Mar 16 14:52:16.133: INFO: Container privileged-container ready: true, restart count 0 Mar 16 14:52:16.133: INFO: apparmor-loader-w2p7f started at 2023-03-16 14:52:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container apparmor-loader ready: true, restart count 0 Mar 16 14:52:16.133: INFO: send-events-9a413a0c-dcdd-40b9-b746-e1662cb824b1 started at 2023-03-16 14:51:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container p ready: true, restart count 0 Mar 16 14:52:16.133: INFO: replace-27982972-fqszk started at 2023-03-16 14:52:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container c ready: true, restart count 0 Mar 16 14:52:16.133: INFO: pod-disruption-failure-ignore-1-j2zz7 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container c ready: false, restart count 0 Mar 16 14:52:16.133: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:16.133: INFO: test-rolling-update-with-lb-69fcddc54f-p8dgp started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:16.133: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:16.133: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:16.133: INFO: pod-disruption-failure-ignore-3-n9wjf started at 2023-03-16 14:52:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container c ready: false, restart count 0 Mar 16 14:52:16.133: INFO: startup-script started at 2023-03-16 14:50:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container startup-script ready: true, restart count 0 Mar 16 14:52:16.133: INFO: test-webserver-a9550890-04c7-4df8-ae3f-3c683c959a30 started at 2023-03-16 14:51:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container test-webserver ready: false, restart count 0 Mar 16 14:52:16.133: INFO: netserver-3 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:16.133: INFO: liveness-de36192b-1862-4140-983a-22bcf2a75063 started at 2023-03-16 14:50:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container agnhost-container ready: false, restart count 3 Mar 16 14:52:16.133: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:16.133: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:16.133: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:16.133: INFO: test-pod started at 2023-03-16 14:52:04 +0000 UTC (0+3 container statuses recorded) Mar 16 14:52:16.133: INFO: Container busybox-1 ready: true, restart count 0 Mar 16 14:52:16.133: INFO: Container busybox-2 ready: true, restart count 0 Mar 16 14:52:16.133: INFO: Container busybox-3 ready: true, restart count 0 Mar 16 14:52:16.133: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:16.133: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:16.133: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:16.133: INFO: pod-disruption-failure-ignore-2-vxzkl started at 2023-03-16 14:52:10 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container c ready: false, restart count 0 Mar 16 14:52:16.133: INFO: pod-disruption-failure-ignore-0-k6595 started at 2023-03-16 14:52:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container c ready: true, restart count 0 Mar 16 14:52:16.133: INFO: pod-disruption-failure-ignore-1-cn28h started at 2023-03-16 14:52:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.133: INFO: Container c ready: false, restart count 0 Mar 16 14:52:16.510: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns STEP: Collecting events from namespace "webhook-markers-5672". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:52:16.51 STEP: Found 0 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:52:16.552 Mar 16 14:52:16.593: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:52:16.593: INFO: Mar 16 14:52:16.636: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:52:16.677: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 660 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:47:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:16.677: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:52:16.722: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:52:16.790: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:52:16.790: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:52:16.790: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:52:16.790: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:52:16.790: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:52:16.790: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:52:16.790: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:16.790: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:16.790: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:16.790: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:52:16.790: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:16.790: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:52:16.997: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:52:16.997: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:17.038: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 8265 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:17.039: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:17.084: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:17.135: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:17.135: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:52:17.135: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:52:17.135: INFO: test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.135: INFO: host-test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:17.135: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:17.135: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.135: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:17.135: INFO: pod-handle-http-request started at 2023-03-16 14:51:36 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:17.135: INFO: Container container-handle-8090-request ready: true, restart count 0 Mar 16 14:52:17.135: INFO: Container container-handle-9090-request ready: true, restart count 0 Mar 16 14:52:17.135: INFO: test-host-network-pod started at 2023-03-16 14:52:07 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:17.135: INFO: Container busybox-1 ready: true, restart count 0 Mar 16 14:52:17.135: INFO: Container busybox-2 ready: true, restart count 0 Mar 16 14:52:17.135: INFO: pod-should-be-evicted4f13af73-bec5-44f3-887b-797d587a0ad5 started at 2023-03-16 14:51:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container bar ready: true, restart count 0 Mar 16 14:52:17.135: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:52:17.135: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container httpd ready: true, restart count 0 Mar 16 14:52:17.135: INFO: busybox-a5a35a8e-81ce-4aed-8ded-436003287c63 started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container busybox ready: true, restart count 0 Mar 16 14:52:17.135: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:17.135: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:52:17.135: INFO: test-rolling-update-with-lb-69fcddc54f-6qbjs started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:17.135: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:17.135: INFO: netserver-0 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.135: INFO: netserver-0 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.135: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:17.135: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:17.135: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:17.135: INFO: pod-client started at 2023-03-16 14:52:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.135: INFO: Container pod-client ready: false, restart count 0 Mar 16 14:52:17.461: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:17.461: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:17.510: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:17.510: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:17.561: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:17.647: INFO: netserver-1 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.647: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.647: INFO: externalname-service-6q82d started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:52:17.647: INFO: busybox-scheduling-b42fe286-a3b5-4398-a9c1-a48f5ac2756d started at 2023-03-16 14:51:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container busybox-scheduling-b42fe286-a3b5-4398-a9c1-a48f5ac2756d ready: true, restart count 0 Mar 16 14:52:17.647: INFO: successful-jobs-history-limit-27982972-5xjm6 started at 2023-03-16 14:52:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container c ready: false, restart count 0 Mar 16 14:52:17.647: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:17.647: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:17.647: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:17.647: INFO: execpodf4mm6 started at 2023-03-16 14:51:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:17.647: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.647: INFO: pause-pod-0 started at 2023-03-16 14:51:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:17.647: INFO: probe-test-b81d4fc4-f22a-4b94-80a5-af7def1bb6a9 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container probe-test-b81d4fc4-f22a-4b94-80a5-af7def1bb6a9 ready: true, restart count 0 Mar 16 14:52:17.647: INFO: pod2 started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container container1 ready: true, restart count 0 Mar 16 14:52:17.647: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:17.647: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container coredns ready: true, restart count 0 Mar 16 14:52:17.647: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:17.647: INFO: execpodnsv2z started at 2023-03-16 14:52:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:17.647: INFO: pod1 started at 2023-03-16 14:52:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container container1 ready: true, restart count 0 Mar 16 14:52:17.647: INFO: pod1 started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:17.647: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:17.647: INFO: verify-service-down-host-exec-pod started at 2023-03-16 14:51:42 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:17.647: INFO: pod1 started at 2023-03-16 14:52:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:17.647: INFO: test-rolling-update-with-lb-69fcddc54f-chjs7 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:17.647: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:17.647: INFO: concurrent-27982972-6527z started at 2023-03-16 14:52:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container c ready: true, restart count 0 Mar 16 14:52:17.647: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:17.647: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:17.937: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:17.937: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:17.997: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 6652 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:17.998: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:18.048: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:52:18.149: INFO: replace-27982971-kn4jb started at 2023-03-16 14:51:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container c ready: true, restart count 0 Mar 16 14:52:18.149: INFO: httpd started at 2023-03-16 14:51:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container httpd ready: true, restart count 0 Mar 16 14:52:18.149: INFO: concurrent-27982971-2w2br started at 2023-03-16 14:51:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container c ready: true, restart count 0 Mar 16 14:52:18.149: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:18.149: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container coredns ready: true, restart count 0 Mar 16 14:52:18.149: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.149: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:52:18.149: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:18.149: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:52:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container agnhost-container ready: false, restart count 0 Mar 16 14:52:18.149: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:18.149: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.149: INFO: dns-test-29bf3d35-0e3a-49a8-a83c-0276b869e7de started at 2023-03-16 14:51:57 +0000 UTC (0+3 container statuses recorded) Mar 16 14:52:18.149: INFO: Container jessie-querier ready: true, restart count 0 Mar 16 14:52:18.149: INFO: Container querier ready: true, restart count 0 Mar 16 14:52:18.149: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.149: INFO: test-grpc-78fcda4d-0543-4b7d-84c7-d1b98379ae7a started at 2023-03-16 14:52:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:18.149: INFO: netserver-2 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.149: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.149: INFO: execpodpplld started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:18.149: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:18.149: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:18.149: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:18.149: INFO: run-test-2 started at 2023-03-16 14:52:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container run-test-2 ready: false, restart count 0 Mar 16 14:52:18.149: INFO: sample-webhook-deployment-7497495989-tmzfq started at 2023-03-16 14:52:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container sample-webhook ready: true, restart count 0 Mar 16 14:52:18.149: INFO: externalname-service-22tpf started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:52:18.149: INFO: kube-dns-autoscaler-7b444c59c9-2q9j6 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.149: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:52:18.483: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:18.483: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:18.550: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 10041 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:52:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:01 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:18.551: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:18.612: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:18.752: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:18.752: INFO: pod-disruption-failure-ignore-2-vxzkl started at 2023-03-16 14:52:10 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container c ready: false, restart count 0 Mar 16 14:52:18.752: INFO: pod-disruption-failure-ignore-0-k6595 started at 2023-03-16 14:52:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container c ready: true, restart count 0 Mar 16 14:52:18.752: INFO: pod-disruption-failure-ignore-1-cn28h started at 2023-03-16 14:52:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container c ready: false, restart count 0 Mar 16 14:52:18.752: INFO: privileged-pod started at 2023-03-16 14:51:57 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:18.752: INFO: Container not-privileged-container ready: true, restart count 0 Mar 16 14:52:18.752: INFO: Container privileged-container ready: true, restart count 0 Mar 16 14:52:18.752: INFO: pod-disruption-failure-ignore-1-j2zz7 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container c ready: false, restart count 0 Mar 16 14:52:18.752: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:18.752: INFO: test-rolling-update-with-lb-69fcddc54f-p8dgp started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:18.752: INFO: replace-27982972-fqszk started at 2023-03-16 14:52:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container c ready: true, restart count 0 Mar 16 14:52:18.752: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:18.752: INFO: hairpin started at 2023-03-16 14:52:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container agnhost-container ready: false, restart count 0 Mar 16 14:52:18.752: INFO: pod-disruption-failure-ignore-3-n9wjf started at 2023-03-16 14:52:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container c ready: false, restart count 0 Mar 16 14:52:18.752: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.752: INFO: netserver-3 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.752: INFO: startup-script started at 2023-03-16 14:50:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container startup-script ready: true, restart count 0 Mar 16 14:52:18.752: INFO: test-webserver-a9550890-04c7-4df8-ae3f-3c683c959a30 started at 2023-03-16 14:51:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container test-webserver ready: true, restart count 0 Mar 16 14:52:18.752: INFO: liveness-de36192b-1862-4140-983a-22bcf2a75063 started at 2023-03-16 14:50:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container agnhost-container ready: false, restart count 3 Mar 16 14:52:18.752: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:18.752: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:18.752: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:18.752: INFO: test-pod started at 2023-03-16 14:52:04 +0000 UTC (0+3 container statuses recorded) Mar 16 14:52:18.752: INFO: Container busybox-1 ready: true, restart count 0 Mar 16 14:52:18.752: INFO: Container busybox-2 ready: true, restart count 0 Mar 16 14:52:18.752: INFO: Container busybox-3 ready: true, restart count 0 Mar 16 14:52:18.752: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:18.752: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:18.752: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:19.066: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:52:19.066 (6.545s) < Exit [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - dump namespaces | framework.go:209 @ 03/16/23 14:52:19.066 (6.545s) > Enter [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - tear down framework | framework.go:206 @ 03/16/23 14:52:19.066 STEP: Destroying namespace "webhook-2754" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:52:19.066 STEP: Destroying namespace "webhook-markers-5672" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:52:19.111 < Exit [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] - tear down framework | framework.go:206 @ 03/16/23 14:52:19.159 (93ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:52:19.159 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:52:19.159 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sReplicaSet\sshould\sserve\sa\sbasic\simage\son\seach\sreplica\swith\sa\sprivate\simage$'
[FAILED] checking pod responses: Told to stop trying after 30.172s. Unexpected final error while getting []pod.response: Controller my-hostname-private-aa234f90-5626-490f-acab-7ef088827140: failed to Get from replica pod my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv: <*errors.StatusError | 0xc0040a0c80>: the server is currently unable to handle the request (get pods my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv) { ErrStatus: code: 503 details: causes: - message: unknown reason: UnexpectedServerResponse kind: pods name: my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv message: the server is currently unable to handle the request (get pods my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv) metadata: {} reason: ServiceUnavailable status: Failure, } Pod status: <v1.PodStatus>: conditions: - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" message: 'containers with unready status: [my-hostname-private-aa234f90-5626-490f-acab-7ef088827140]' reason: ContainersNotReady status: "False" type: Ready - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" message: 'containers with unready status: [my-hostname-private-aa234f90-5626-490f-acab-7ef088827140]' reason: ContainersNotReady status: "False" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" status: "True" type: PodScheduled containerStatuses: - image: gcr.io/k8s-authenticated-test/agnhost:2.6 imageID: "" lastState: {} name: my-hostname-private-aa234f90-5626-490f-acab-7ef088827140 ready: false restartCount: 0 started: false state: waiting: reason: ContainerCreating hostIP: 10.40.0.3 phase: Pending qosClass: BestEffort startTime: "2023-03-16T14:56:15Z" In [It] at: test/e2e/apps/replica_set.go:228 @ 03/16/23 14:56:52.193from junit_01.xml
> Enter [BeforeEach] [sig-apps] ReplicaSet - set up framework | framework.go:191 @ 03/16/23 14:56:15.379 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:56:15.379 Mar 16 14:56:15.379: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename replicaset - test/e2e/framework/framework.go:250 @ 03/16/23 14:56:15.38 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:56:15.513 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:56:15.596 < Exit [BeforeEach] [sig-apps] ReplicaSet - set up framework | framework.go:191 @ 03/16/23 14:56:15.677 (298ms) > Enter [BeforeEach] [sig-apps] ReplicaSet - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:56:15.677 < Exit [BeforeEach] [sig-apps] ReplicaSet - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:56:15.677 (0s) > Enter [It] should serve a basic image on each replica with a private image - test/e2e/apps/replica_set.go:115 @ 03/16/23 14:56:15.677 Mar 16 14:56:15.677: INFO: Creating ReplicaSet my-hostname-private-aa234f90-5626-490f-acab-7ef088827140 Mar 16 14:56:15.768: INFO: Pod name my-hostname-private-aa234f90-5626-490f-acab-7ef088827140: Found 1 pods out of 1 Mar 16 14:56:15.768: INFO: Ensuring a pod for ReplicaSet "my-hostname-private-aa234f90-5626-490f-acab-7ef088827140" is running Mar 16 14:56:22.020: INFO: Pod "my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv" is running (conditions: [{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:56:15 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:56:15 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [my-hostname-private-aa234f90-5626-490f-acab-7ef088827140]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:56:15 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [my-hostname-private-aa234f90-5626-490f-acab-7ef088827140]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:56:15 +0000 UTC Reason: Message:}]) Mar 16 14:56:22.020: INFO: Trying to dial the pod STEP: trying to dial each unique pod - test/e2e/framework/pod/wait.go:550 @ 03/16/23 14:56:22.02 Mar 16 14:56:52.193: INFO: Failed inside E2E framework: k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodsResponding({0x7f0d94412bf8, 0xc004327bf0}, {0x729b828?, 0xc00231ba00}, {0xc004078f20, 0xf}, {0xc004163080, 0x38}, 0x1, 0x1bf08eb000, ...) test/e2e/framework/pod/wait.go:625 +0x3ef k8s.io/kubernetes/test/e2e/apps.testReplicaSetServeImageOrFail({0x7f0d94412bf8, 0xc004327bf0}, 0xc000d520f0, {0x6a8cb17, 0x7}, {0xc004354450, 0x29}) test/e2e/apps/replica_set.go:228 +0xac5 k8s.io/kubernetes/test/e2e/apps.glob..func9.2({0x7f0d94412bf8, 0xc004327bf0}) test/e2e/apps/replica_set.go:119 +0x1b9 [FAILED] checking pod responses: Told to stop trying after 30.172s. Unexpected final error while getting []pod.response: Controller my-hostname-private-aa234f90-5626-490f-acab-7ef088827140: failed to Get from replica pod my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv: <*errors.StatusError | 0xc0040a0c80>: the server is currently unable to handle the request (get pods my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv) { ErrStatus: code: 503 details: causes: - message: unknown reason: UnexpectedServerResponse kind: pods name: my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv message: the server is currently unable to handle the request (get pods my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv) metadata: {} reason: ServiceUnavailable status: Failure, } Pod status: <v1.PodStatus>: conditions: - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" message: 'containers with unready status: [my-hostname-private-aa234f90-5626-490f-acab-7ef088827140]' reason: ContainersNotReady status: "False" type: Ready - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" message: 'containers with unready status: [my-hostname-private-aa234f90-5626-490f-acab-7ef088827140]' reason: ContainersNotReady status: "False" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2023-03-16T14:56:15Z" status: "True" type: PodScheduled containerStatuses: - image: gcr.io/k8s-authenticated-test/agnhost:2.6 imageID: "" lastState: {} name: my-hostname-private-aa234f90-5626-490f-acab-7ef088827140 ready: false restartCount: 0 started: false state: waiting: reason: ContainerCreating hostIP: 10.40.0.3 phase: Pending qosClass: BestEffort startTime: "2023-03-16T14:56:15Z" In [It] at: test/e2e/apps/replica_set.go:228 @ 03/16/23 14:56:52.193 < Exit [It] should serve a basic image on each replica with a private image - test/e2e/apps/replica_set.go:115 @ 03/16/23 14:56:52.193 (36.516s) > Enter [AfterEach] [sig-apps] ReplicaSet - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:56:52.193 Mar 16 14:56:52.193: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-apps] ReplicaSet - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:56:52.239 (46ms) > Enter [DeferCleanup (Each)] [sig-apps] ReplicaSet - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:56:52.24 < Exit [DeferCleanup (Each)] [sig-apps] ReplicaSet - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:56:52.24 (0s) > Enter [DeferCleanup (Each)] [sig-apps] ReplicaSet - dump namespaces | framework.go:209 @ 03/16/23 14:56:52.24 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:56:52.24 STEP: Collecting events from namespace "replicaset-9198". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:56:52.24 STEP: Found 6 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:56:52.323 Mar 16 14:56:52.323: INFO: At 2023-03-16 14:56:15 +0000 UTC - event for my-hostname-private-aa234f90-5626-490f-acab-7ef088827140: {replicaset-controller } SuccessfulCreate: Created pod: my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv Mar 16 14:56:52.323: INFO: At 2023-03-16 14:56:15 +0000 UTC - event for my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv: {default-scheduler } Scheduled: Successfully assigned replicaset-9198/my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv to bootstrap-e2e-minion-group-z4ns Mar 16 14:56:52.323: INFO: At 2023-03-16 14:56:16 +0000 UTC - event for my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv: {kubelet bootstrap-e2e-minion-group-z4ns} Pulling: Pulling image "gcr.io/k8s-authenticated-test/agnhost:2.6" Mar 16 14:56:52.323: INFO: At 2023-03-16 14:56:20 +0000 UTC - event for my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Successfully pulled image "gcr.io/k8s-authenticated-test/agnhost:2.6" in 3.642117807s (3.642146188s including waiting) Mar 16 14:56:52.323: INFO: At 2023-03-16 14:56:20 +0000 UTC - event for my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container my-hostname-private-aa234f90-5626-490f-acab-7ef088827140 Mar 16 14:56:52.323: INFO: At 2023-03-16 14:56:20 +0000 UTC - event for my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container my-hostname-private-aa234f90-5626-490f-acab-7ef088827140 Mar 16 14:56:52.365: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:56:52.365: INFO: my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv bootstrap-e2e-minion-group-z4ns Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:15 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:20 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:20 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:15 +0000 UTC }] Mar 16 14:56:52.365: INFO: Mar 16 14:56:52.465: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:56:52.508: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 12046 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:52:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:56:52.508: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:56:52.556: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:56:52.617: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:56:52.617: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:56:52.617: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:56:52.617: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:56:52.617: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:56:52.617: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:56:52.617: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:56:52.617: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:56:52.617: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:56:52.617: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:56:52.617: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:52.617: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:56:52.834: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:56:52.834: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:56:52.880: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 18970 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:54:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:55:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:54:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:54:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:54:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:54:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:54:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:54:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:54:59 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:55:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:55:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:55:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:55:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:56:52.881: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:56:52.932: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:56:53.084: INFO: pause-pod-54b45db9d8-857d9 started at 2023-03-16 14:56:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container agnhost-pause ready: true, restart count 0 Mar 16 14:56:53.084: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:56:53.084: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:56:53.084: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:56:53.084: INFO: netserver-0 started at 2023-03-16 14:56:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: false, restart count 0 Mar 16 14:56:53.084: INFO: netserver-0 started at 2023-03-16 14:55:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: test-ss-1 started at 2023-03-16 14:56:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:56:53.084: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:56:53.084: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:56:53.084: INFO: execpod7b67j started at 2023-03-16 14:53:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.084: INFO: netserver-0 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: netserver-0 started at 2023-03-16 14:52:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:56:53.084: INFO: netserver-0 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: service-headless-45t5c started at 2023-03-16 14:55:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container service-headless ready: true, restart count 0 Mar 16 14:56:53.084: INFO: rs-ngps7 started at 2023-03-16 14:55:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container donothing ready: false, restart count 0 Mar 16 14:56:53.084: INFO: netserver-0 started at 2023-03-16 14:54:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: test-container-pod started at 2023-03-16 14:55:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: service-headless-toggled-qgdzs started at 2023-03-16 14:55:42 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container service-headless-toggled ready: true, restart count 0 Mar 16 14:56:53.084: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:56:53.084: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container httpd ready: true, restart count 0 Mar 16 14:56:53.084: INFO: webserver-pod started at 2023-03-16 14:54:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container agnhost ready: false, restart count 0 Mar 16 14:56:53.084: INFO: pause-pod-1 started at 2023-03-16 14:55:42 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.084: INFO: service-deletion-snn22 started at 2023-03-16 14:56:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container service-deletion ready: true, restart count 0 Mar 16 14:56:53.084: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.084: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:56:53.084: INFO: affinity-nodeport-timeout-gwrjk started at 2023-03-16 14:55:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Mar 16 14:56:53.084: INFO: update-demo-nautilus-hkt7h started at 2023-03-16 14:56:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container update-demo ready: true, restart count 0 Mar 16 14:56:53.084: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.084: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:56:53.335: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:56:53.335: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:56:53.384: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 18960 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:55:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:55:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:55:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:55:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:55:03 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:55:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:55:37 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:55:37 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:55:37 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:55:37 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:56:53.384: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:56:53.430: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:54:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: affinity-nodeport-timeout-q4nbz started at 2023-03-16 14:55:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:56:53.515: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:56:53.515: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:56:53.515: INFO: service-headless-7cgt9 started at 2023-03-16 14:55:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container service-headless ready: true, restart count 0 Mar 16 14:56:53.515: INFO: slave started at 2023-03-16 14:56:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container cntr ready: true, restart count 0 Mar 16 14:56:53.515: INFO: echo-sourceip started at 2023-03-16 14:56:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.515: INFO: verify-service-down-host-exec-pod started at 2023-03-16 14:56:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.515: INFO: test-container-pod started at 2023-03-16 14:54:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container coredns ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:55:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: private started at 2023-03-16 14:56:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container cntr ready: false, restart count 0 Mar 16 14:56:53.515: INFO: master started at 2023-03-16 14:56:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container cntr ready: true, restart count 0 Mar 16 14:56:53.515: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:56:53.515: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.515: INFO: host-test-container-pod started at 2023-03-16 14:56:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:56:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: false, restart count 0 Mar 16 14:56:53.515: INFO: kube-proxy-mode-detector started at 2023-03-16 14:55:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.515: INFO: update-demo-nautilus-2lchd started at 2023-03-16 14:56:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container update-demo ready: true, restart count 0 Mar 16 14:56:53.515: INFO: nodeport-update-service-czr4d started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:56:53.515: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.515: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.789: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:56:53.789: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:56:53.831: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 21581 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:55:07 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:56:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:55:07 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:07 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:07 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:07 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:55:07 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:55:07 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:55:07 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:56:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:56:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:56:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:56:26 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:56:53.831: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:56:53.876: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:56:53.933: INFO: rs-g465c started at 2023-03-16 14:55:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container donothing ready: false, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:54:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:56:53.933: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:56:53.933: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: host-test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.933: INFO: execpod-affinityvnhd6 started at 2023-03-16 14:55:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:55:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:56:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: false, restart count 0 Mar 16 14:56:53.933: INFO: busybox-d2f00579-9a9e-41fa-8711-df9ea7313998 started at 2023-03-16 14:56:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container busybox ready: true, restart count 0 Mar 16 14:56:53.933: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:56:53.933: INFO: tester started at 2023-03-16 14:56:46 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container tester ready: true, restart count 0 Mar 16 14:56:53.933: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:56:53.933: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container coredns ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:54:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: startup-fef7a54a-b1aa-4158-abf1-ecc20d8b7201 started at 2023-03-16 14:56:32 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container busybox ready: false, restart count 0 Mar 16 14:56:53.933: INFO: pod-ready started at 2023-03-16 14:56:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container pod-readiness-gate ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: pod-qos-class-87512bc2-3657-4495-98fe-e7845dbc42f2 started at 2023-03-16 14:56:32 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container agnhost ready: false, restart count 0 Mar 16 14:56:53.933: INFO: service-headless-vqw8q started at 2023-03-16 14:55:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container service-headless ready: true, restart count 0 Mar 16 14:56:53.933: INFO: service-headless-toggled-sr685 started at 2023-03-16 14:55:42 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container service-headless-toggled ready: true, restart count 0 Mar 16 14:56:53.933: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:56:53.933: INFO: test-container-pod started at 2023-03-16 14:56:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: netserver-2 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:53.933: INFO: test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:53.933: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.156: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:56:54.156: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:56:54.199: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 21681 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:55:11 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:56:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:11 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:11 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:55:11 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:55:11 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:55:11 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:55:11 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:55:11 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:56:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:56:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:56:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:56:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:2110879,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:56:54.199: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:56:54.242: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:56:54.296: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:56:54.296: INFO: test-container-pod started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:54:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:56:53 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container agnhost-container ready: false, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: nodeport-update-service-d27z2 started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:56:54.296: INFO: my-hostname-private-aa234f90-5626-490f-acab-7ef088827140-cdhnv started at 2023-03-16 14:56:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container my-hostname-private-aa234f90-5626-490f-acab-7ef088827140 ready: true, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:56:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: false, restart count 0 Mar 16 14:56:54.296: INFO: pause-pod-54b45db9d8-trlvq started at 2023-03-16 14:56:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container agnhost-pause ready: true, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:55:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: test-container-pod started at 2023-03-16 14:53:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: service-headless-toggled-8wpnm started at 2023-03-16 14:55:42 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container service-headless-toggled ready: true, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:54:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: test-container-pod started at 2023-03-16 14:54:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: affinity-nodeport-timeout-chlwm started at 2023-03-16 14:55:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 Mar 16 14:56:54.296: INFO: webserver-pod started at 2023-03-16 14:55:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container agnhost ready: false, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: sample-apiserver-deployment-6dfd6dfd5b-lkfh2 started at 2023-03-16 14:56:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:56:54.296: INFO: Container etcd ready: true, restart count 0 Mar 16 14:56:54.296: INFO: Container sample-apiserver ready: false, restart count 0 Mar 16 14:56:54.296: INFO: test-ss-0 started at 2023-03-16 14:55:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: kube-dns-autoscaler-7b444c59c9-wglx5 started at 2023-03-16 14:53:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:56:54.296: INFO: startup-92f50ab5-2a9b-430b-a1c1-f717132f92c7 started at 2023-03-16 14:56:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container busybox ready: false, restart count 0 Mar 16 14:56:54.296: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:56:54.296: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:56:54.296: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:56:54.296: INFO: netserver-3 started at 2023-03-16 14:54:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container webserver ready: true, restart count 0 Mar 16 14:56:54.296: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:56:54.296: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:56:54.532: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:56:54.532 (2.292s) < Exit [DeferCleanup (Each)] [sig-apps] ReplicaSet - dump namespaces | framework.go:209 @ 03/16/23 14:56:54.532 (2.292s) > Enter [DeferCleanup (Each)] [sig-apps] ReplicaSet - tear down framework | framework.go:206 @ 03/16/23 14:56:54.532 STEP: Destroying namespace "replicaset-9198" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:56:54.532 < Exit [DeferCleanup (Each)] [sig-apps] ReplicaSet - tear down framework | framework.go:206 @ 03/16/23 14:56:54.576 (44ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:56:54.576 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:56:54.576 (0s)
Find responses mentions in log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-apps\]\sReplicationController\sshould\sserve\sa\sbasic\simage\son\seach\sreplica\swith\sa\sprivate\simage$'
[FAILED] checking pod responses: Told to stop trying after 30.141s. Unexpected final error while getting []pod.response: Controller my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac: failed to Get from replica pod my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6: <*errors.StatusError | 0xc0040f0320>: the server is currently unable to handle the request (get pods my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6) { ErrStatus: code: 503 details: causes: - message: unknown reason: UnexpectedServerResponse kind: pods name: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 message: the server is currently unable to handle the request (get pods my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6) metadata: {} reason: ServiceUnavailable status: Failure, } Pod status: <v1.PodStatus>: conditions: - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" message: 'containers with unready status: [my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac]' reason: ContainersNotReady status: "False" type: Ready - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" message: 'containers with unready status: [my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac]' reason: ContainersNotReady status: "False" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" status: "True" type: PodScheduled containerStatuses: - image: gcr.io/k8s-authenticated-test/agnhost:2.6 imageID: "" lastState: {} name: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac ready: false restartCount: 0 started: false state: waiting: reason: ContainerCreating hostIP: 10.40.0.5 phase: Pending qosClass: BestEffort startTime: "2023-03-16T14:53:51Z" In [It] at: test/e2e/apps/rc.go:531 @ 03/16/23 14:54:39.757from junit_01.xml
> Enter [BeforeEach] [sig-apps] ReplicationController - set up framework | framework.go:191 @ 03/16/23 14:53:49.255 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:53:49.255 Mar 16 14:53:49.255: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename replication-controller - test/e2e/framework/framework.go:250 @ 03/16/23 14:53:49.256 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:53:49.651 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:53:49.773 < Exit [BeforeEach] [sig-apps] ReplicationController - set up framework | framework.go:191 @ 03/16/23 14:53:49.929 (674ms) > Enter [BeforeEach] [sig-apps] ReplicationController - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:53:49.929 < Exit [BeforeEach] [sig-apps] ReplicationController - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:53:49.929 (0s) > Enter [BeforeEach] [sig-apps] ReplicationController - test/e2e/apps/rc.go:59 @ 03/16/23 14:53:49.929 < Exit [BeforeEach] [sig-apps] ReplicationController - test/e2e/apps/rc.go:59 @ 03/16/23 14:53:49.929 (0s) > Enter [It] should serve a basic image on each replica with a private image - test/e2e/apps/rc.go:73 @ 03/16/23 14:53:49.929 STEP: Creating replication controller my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac - test/e2e/apps/rc.go:493 @ 03/16/23 14:53:49.929 Mar 16 14:53:50.106: INFO: Pod name my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac: Found 0 pods out of 1 Mar 16 14:53:55.148: INFO: Pod name my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac: Found 1 pods out of 1 Mar 16 14:53:55.148: INFO: Ensuring all pods for ReplicationController "my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac" are running Mar 16 14:54:09.616: INFO: Pod "my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6" is running (conditions: [{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:53:51 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:53:51 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:53:51 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-03-16 14:53:51 +0000 UTC Reason: Message:}]) Mar 16 14:54:09.616: INFO: Trying to dial the pod STEP: trying to dial each unique pod - test/e2e/framework/pod/wait.go:550 @ 03/16/23 14:54:09.616 Mar 16 14:54:39.757: INFO: Failed inside E2E framework: k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodsResponding({0x7f1be872b098, 0xc004ab5f80}, {0x729b828?, 0xc002e75860}, {0xc00498d340, 0x1b}, {0xc0049734c0, 0x38}, 0x1, 0x1bf08eb000, ...) test/e2e/framework/pod/wait.go:625 +0x3ef k8s.io/kubernetes/test/e2e/apps.TestReplicationControllerServeImageOrFail({0x7f1be872b098, 0xc004ab5f80}, 0xc000954000, {0x6a8cb17, 0x7}, {0xc004abf2f0, 0x29}) test/e2e/apps/rc.go:531 +0xaa5 k8s.io/kubernetes/test/e2e/apps.glob..func8.3({0x7f1be872b098, 0xc004ab5f80}) test/e2e/apps/rc.go:77 +0x1b9 [FAILED] checking pod responses: Told to stop trying after 30.141s. Unexpected final error while getting []pod.response: Controller my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac: failed to Get from replica pod my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6: <*errors.StatusError | 0xc0040f0320>: the server is currently unable to handle the request (get pods my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6) { ErrStatus: code: 503 details: causes: - message: unknown reason: UnexpectedServerResponse kind: pods name: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 message: the server is currently unable to handle the request (get pods my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6) metadata: {} reason: ServiceUnavailable status: Failure, } Pod status: <v1.PodStatus>: conditions: - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" message: 'containers with unready status: [my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac]' reason: ContainersNotReady status: "False" type: Ready - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" message: 'containers with unready status: [my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac]' reason: ContainersNotReady status: "False" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2023-03-16T14:53:51Z" status: "True" type: PodScheduled containerStatuses: - image: gcr.io/k8s-authenticated-test/agnhost:2.6 imageID: "" lastState: {} name: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac ready: false restartCount: 0 started: false state: waiting: reason: ContainerCreating hostIP: 10.40.0.5 phase: Pending qosClass: BestEffort startTime: "2023-03-16T14:53:51Z" In [It] at: test/e2e/apps/rc.go:531 @ 03/16/23 14:54:39.757 < Exit [It] should serve a basic image on each replica with a private image - test/e2e/apps/rc.go:73 @ 03/16/23 14:54:39.757 (49.828s) > Enter [AfterEach] [sig-apps] ReplicationController - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:54:39.758 Mar 16 14:54:39.758: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-apps] ReplicationController - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:54:39.799 (42ms) > Enter [DeferCleanup (Each)] [sig-apps] ReplicationController - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:54:39.799 < Exit [DeferCleanup (Each)] [sig-apps] ReplicationController - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:54:39.799 (0s) > Enter [DeferCleanup (Each)] [sig-apps] ReplicationController - dump namespaces | framework.go:209 @ 03/16/23 14:54:39.799 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:54:39.799 STEP: Collecting events from namespace "replication-controller-9426". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:54:39.799 STEP: Found 6 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:54:39.841 Mar 16 14:54:39.841: INFO: At 2023-03-16 14:53:51 +0000 UTC - event for my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac: {replication-controller } SuccessfulCreate: Created pod: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 Mar 16 14:54:39.841: INFO: At 2023-03-16 14:53:51 +0000 UTC - event for my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6: {default-scheduler } Scheduled: Successfully assigned replication-controller-9426/my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 to bootstrap-e2e-minion-group-2p9b Mar 16 14:54:39.841: INFO: At 2023-03-16 14:54:01 +0000 UTC - event for my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6: {kubelet bootstrap-e2e-minion-group-2p9b} Pulling: Pulling image "gcr.io/k8s-authenticated-test/agnhost:2.6" Mar 16 14:54:39.841: INFO: At 2023-03-16 14:54:08 +0000 UTC - event for my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6: {kubelet bootstrap-e2e-minion-group-2p9b} Pulled: Successfully pulled image "gcr.io/k8s-authenticated-test/agnhost:2.6" in 6.656726039s (6.656738404s including waiting) Mar 16 14:54:39.841: INFO: At 2023-03-16 14:54:08 +0000 UTC - event for my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6: {kubelet bootstrap-e2e-minion-group-2p9b} Created: Created container my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac Mar 16 14:54:39.841: INFO: At 2023-03-16 14:54:08 +0000 UTC - event for my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6: {kubelet bootstrap-e2e-minion-group-2p9b} Started: Started container my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac Mar 16 14:54:39.882: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:54:39.882: INFO: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 bootstrap-e2e-minion-group-2p9b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:53:51 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:54:09 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:54:09 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:53:51 +0000 UTC }] Mar 16 14:54:39.882: INFO: Mar 16 14:54:39.972: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:54:40.014: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 12046 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:52:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:40.014: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:54:40.058: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:54:40.106: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:54:40.106: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:54:40.106: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:54:40.106: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:40.106: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:40.106: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:40.106: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:54:40.106: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:54:40.106: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:54:40.106: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:54:40.106: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.106: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:54:40.348: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:54:40.348: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:40.391: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 16125 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:40.392: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:40.444: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:54:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:40.523: INFO: execpod-affinityn8trg started at 2023-03-16 14:54:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:40.523: INFO: test-deployment-hskb9-5994cf9475-slxjz started at 2023-03-16 14:54:35 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container httpd ready: true, restart count 0 Mar 16 14:54:40.523: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container httpd ready: true, restart count 0 Mar 16 14:54:40.523: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:54:40.523: INFO: webserver-pod started at 2023-03-16 14:54:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container agnhost ready: false, restart count 0 Mar 16 14:54:40.523: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:54:40.523: INFO: affinity-nodeport-transition-g77pz started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:40.523: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:40.523: INFO: up-down-3-6hgmw started at 2023-03-16 14:54:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container up-down-3 ready: true, restart count 0 Mar 16 14:54:40.523: INFO: dns-test-9070105a-7f9d-472d-ab82-8edbd2db49d2 started at 2023-03-16 14:54:36 +0000 UTC (0+3 container statuses recorded) Mar 16 14:54:40.523: INFO: Container jessie-querier ready: true, restart count 0 Mar 16 14:54:40.523: INFO: Container querier ready: true, restart count 0 Mar 16 14:54:40.523: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:40.523: INFO: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac ready: true, restart count 0 Mar 16 14:54:40.523: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:40.523: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:40.523: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:40.523: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:40.523: INFO: execpod7b67j started at 2023-03-16 14:53:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:40.523: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:40.523: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:54:40.523: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:52:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:40.523: INFO: all-pods-removed-tv8rl started at 2023-03-16 14:54:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container c ready: true, restart count 0 Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:40.523: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:40.523: INFO: netserver-0 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:40.523: INFO: all-pods-removed-z7m42 started at 2023-03-16 14:54:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container c ready: true, restart count 0 Mar 16 14:54:40.523: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:40.523: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:40.791: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:40.791: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:40.842: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:40.842: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:40.888: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:41.084: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:41.084: INFO: client-containers-cc8d3fc6-a0c7-4453-8d92-fe7d21a06977 started at 2023-03-16 14:54:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container agnhost-container ready: false, restart count 0 Mar 16 14:54:41.084: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:41.084: INFO: sysctl-305ef13f-faab-40e7-9473-ad1bc288e027 started at 2023-03-16 14:54:38 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container test-container ready: false, restart count 0 Mar 16 14:54:41.084: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: up-down-3-zhgsc started at 2023-03-16 14:54:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container up-down-3 ready: true, restart count 0 Mar 16 14:54:41.084: INFO: pod-client started at 2023-03-16 14:54:19 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container pod-client ready: true, restart count 0 Mar 16 14:54:41.084: INFO: test-ss-0 started at 2023-03-16 14:54:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: nodeport-update-service-czr4d started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:54:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:41.084: INFO: exceed-active-deadline-5d7fv started at 2023-03-16 14:54:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container c ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:41.084: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:41.084: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:41.084: INFO: alpine-nnp-false-1c058660-aa50-495f-acb1-262863728a83 started at 2023-03-16 14:54:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container alpine-nnp-false-1c058660-aa50-495f-acb1-262863728a83 ready: false, restart count 0 Mar 16 14:54:41.084: INFO: affinity-clusterip-transition-7prln started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:41.084: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container coredns ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: test-container-pod started at 2023-03-16 14:54:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:41.084: INFO: netserver-1 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:41.084: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.745: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:42.745: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:42.788: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 15288 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:42.788: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:42.840: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:54:42.920: INFO: host-test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:42.920: INFO: pod2 started at 2023-03-16 14:54:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:42.920: INFO: pod3 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:42.920: INFO: execpod-affinityztffd started at 2023-03-16 14:53:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:42.920: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:54:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: affinity-clusterip-transition-28fjr started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:42.920: INFO: pod-server-1 started at 2023-03-16 14:54:22 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:54:42.920: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container coredns ready: true, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: affinity-nodeport-transition-w6wn6 started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 16 14:54:42.920: INFO: pod1 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:42.920: INFO: pod-server-2 started at 2023-03-16 14:54:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost-container ready: false, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:42.920: INFO: e2e-host-exec started at 2023-03-16 14:54:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container e2e-host-exec ready: true, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:54:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: test-grpc-78fcda4d-0543-4b7d-84c7-d1b98379ae7a started at 2023-03-16 14:52:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:42.920: INFO: netserver-2 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:42.920: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:42.920: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:42.920: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:42.920: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:42.920: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.230: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:43.230: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:43.285: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 15287 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:43.286: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:43.337: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:43.422: INFO: ephemeral-containers-target-pod started at 2023-03-16 14:53:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container test-container-1 ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:54:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: test-container-pod started at 2023-03-16 14:54:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:43.422: INFO: kube-dns-autoscaler-7b444c59c9-wglx5 started at 2023-03-16 14:53:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:54:43.422: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:43.422: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:43.422: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:43.422: INFO: affinity-clusterip-transition-p88jm started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:43.422: INFO: affinity-nodeport-transition-j5t9c started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:54:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:43.422: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:43.422: INFO: exceed-active-deadline-8c5t8 started at 2023-03-16 14:54:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container c ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: test-container-pod started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:54:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: kube-proxy-mode-detector started at 2023-03-16 14:54:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:43.422: INFO: verify-service-up-exec-pod-n876n started at 2023-03-16 14:54:19 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:43.422: INFO: nodeport-update-service-d27z2 started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:54:43.422: INFO: up-down-3-gdlxs started at 2023-03-16 14:54:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container up-down-3 ready: true, restart count 0 Mar 16 14:54:43.422: INFO: webserver-pod started at 2023-03-16 14:53:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: test-container-pod started at 2023-03-16 14:53:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:43.422: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:43.422: INFO: netserver-3 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:43.422: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:44.109: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:54:44.109 (4.309s) < Exit [DeferCleanup (Each)] [sig-apps] ReplicationController - dump namespaces | framework.go:209 @ 03/16/23 14:54:44.109 (4.309s) > Enter [DeferCleanup (Each)] [sig-apps] ReplicationController - tear down framework | framework.go:206 @ 03/16/23 14:54:44.109 STEP: Destroying namespace "replication-controller-9426" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:54:44.109 < Exit [DeferCleanup (Each)] [sig-apps] ReplicationController - tear down framework | framework.go:206 @ 03/16/23 14:54:44.167 (58ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:54:44.167 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:54:44.167 (0s)
Find responses mentions in log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sPort\sforwarding\sWith\sa\sserver\slistening\son\slocalhost\sshould\ssupport\sforwarding\sover\swebsockets$'
[FAILED] Failed to open websocket to wss://34.105.125.69/api/v1/namespaces/port-forwarding-6874/pods/pfpod/portforward?ports=80: websocket.Dial wss://34.105.125.69/api/v1/namespaces/port-forwarding-6874/pods/pfpod/portforward?ports=80: bad status In [It] at: test/e2e/kubectl/portforward.go:385 @ 03/16/23 14:50:14.069from junit_01.xml
> Enter [BeforeEach] [sig-cli] Kubectl Port forwarding - set up framework | framework.go:191 @ 03/16/23 14:49:35.09 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:49:35.09 Mar 16 14:49:35.090: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename port-forwarding - test/e2e/framework/framework.go:250 @ 03/16/23 14:49:35.092 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:49:35.238 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:49:35.338 < Exit [BeforeEach] [sig-cli] Kubectl Port forwarding - set up framework | framework.go:191 @ 03/16/23 14:49:35.423 (333ms) > Enter [BeforeEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:49:35.423 < Exit [BeforeEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:49:35.423 (0s) > Enter [It] should support forwarding over websockets - test/e2e/kubectl/portforward.go:492 @ 03/16/23 14:49:35.423 Mar 16 14:49:35.423: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Creating the pod - test/e2e/kubectl/portforward.go:366 @ 03/16/23 14:49:35.424 [FAILED] Failed to open websocket to wss://34.105.125.69/api/v1/namespaces/port-forwarding-6874/pods/pfpod/portforward?ports=80: websocket.Dial wss://34.105.125.69/api/v1/namespaces/port-forwarding-6874/pods/pfpod/portforward?ports=80: bad status In [It] at: test/e2e/kubectl/portforward.go:385 @ 03/16/23 14:50:14.069 < Exit [It] should support forwarding over websockets - test/e2e/kubectl/portforward.go:492 @ 03/16/23 14:50:14.069 (38.645s) > Enter [AfterEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:50:14.069 Mar 16 14:50:14.069: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-cli] Kubectl Port forwarding - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:50:14.128 (59ms) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:50:14.128 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:50:14.128 (0s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - dump namespaces | framework.go:209 @ 03/16/23 14:50:14.128 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:50:14.128 STEP: Collecting events from namespace "port-forwarding-6874". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:50:14.128 STEP: Found 7 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:50:14.237 Mar 16 14:50:14.237: INFO: At 2023-03-16 14:49:35 +0000 UTC - event for pfpod: {default-scheduler } Scheduled: Successfully assigned port-forwarding-6874/pfpod to bootstrap-e2e-minion-group-2p9b Mar 16 14:50:14.237: INFO: At 2023-03-16 14:49:37 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-2p9b} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:50:14.237: INFO: At 2023-03-16 14:49:37 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-2p9b} Created: Created container readiness Mar 16 14:50:14.237: INFO: At 2023-03-16 14:49:37 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-2p9b} Started: Started container readiness Mar 16 14:50:14.237: INFO: At 2023-03-16 14:49:37 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-2p9b} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:50:14.237: INFO: At 2023-03-16 14:49:37 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-2p9b} Created: Created container portforwardtester Mar 16 14:50:14.237: INFO: At 2023-03-16 14:49:37 +0000 UTC - event for pfpod: {kubelet bootstrap-e2e-minion-group-2p9b} Started: Started container portforwardtester Mar 16 14:50:14.290: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:50:14.290: INFO: pfpod bootstrap-e2e-minion-group-2p9b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:49:35 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:49:42 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:49:42 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:49:35 +0000 UTC }] Mar 16 14:50:14.291: INFO: Mar 16 14:50:14.489: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:50:14.537: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 660 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:47:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:14.537: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:50:14.584: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:50:14.658: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:50:14.658: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:50:14.658: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:50:14.658: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:14.658: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:14.658: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:14.658: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:50:14.658: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:50:14.658: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:50:14.658: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:50:14.658: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:14.658: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:50:14.888: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:50:14.888: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:14.940: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 4261 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:49:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:14.940: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:14.987: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:15.142: INFO: ss2-1 started at 2023-03-16 14:50:03 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:50:15.142: INFO: up-down-1-bs2xk started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container up-down-1 ready: true, restart count 0 Mar 16 14:50:15.142: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:15.142: INFO: test-container-pod started at 2023-03-16 14:50:04 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: dns-test-0667a486-49b4-4b08-9ca9-fd90309002a5 started at 2023-03-16 14:50:00 +0000 UTC (0+3 container statuses recorded) Mar 16 14:50:15.142: INFO: Container jessie-querier ready: true, restart count 0 Mar 16 14:50:15.142: INFO: Container querier ready: true, restart count 0 Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: pod-2 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container donothing ready: true, restart count 0 Mar 16 14:50:15.142: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container httpd ready: true, restart count 0 Mar 16 14:50:15.142: INFO: netserver-0 started at 2023-03-16 14:49:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:50:15.142: INFO: busybox-a5a35a8e-81ce-4aed-8ded-436003287c63 started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container busybox ready: true, restart count 0 Mar 16 14:50:15.142: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:49:53 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:15.142: INFO: verify-service-up-exec-pod-b5kdb started at 2023-03-16 14:49:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:15.142: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:50:15.142: INFO: startup-6a3107dc-925f-44e7-a222-640c4b0abd9e started at 2023-03-16 14:49:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container busybox ready: true, restart count 0 Mar 16 14:50:15.142: INFO: netserver-0 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: test-webserver-ffa602b2-b01a-48fb-94e6-32dd96b9bc8e started at 2023-03-16 14:49:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container test-webserver ready: false, restart count 0 Mar 16 14:50:15.142: INFO: netserver-0 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: simpletest.rc-w2k9t started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:15.142: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:15.142: INFO: pod-1 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container donothing ready: true, restart count 0 Mar 16 14:50:15.142: INFO: test-container-pod started at 2023-03-16 14:49:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: sample-webhook-deployment-7497495989-n2jht started at 2023-03-16 14:50:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container sample-webhook ready: true, restart count 0 Mar 16 14:50:15.142: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:15.142: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:15.142: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:15.142: INFO: simpletest.rc-z995c started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:15.142: INFO: test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.142: INFO: pfpod started at 2023-03-16 14:49:35 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:15.142: INFO: Container portforwardtester ready: true, restart count 0 Mar 16 14:50:15.142: INFO: Container readiness ready: true, restart count 0 Mar 16 14:50:15.142: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:15.142: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:50:15.142: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:50:15.142: INFO: execpod-affinityp5ndd started at 2023-03-16 14:49:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:15.142: INFO: host-test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.142: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:15.509: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:15.509: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:15.559: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 4339 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:15.560: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:15.733: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:15.799: INFO: pod-server-2 started at 2023-03-16 14:50:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:15.799: INFO: affinity-nodeport-88b8w started at 2023-03-16 14:48:54 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container affinity-nodeport ready: true, restart count 0 Mar 16 14:50:15.799: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container coredns ready: true, restart count 0 Mar 16 14:50:15.799: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:50:15.799: INFO: alpine-nnp-true-39d13e6b-e843-403a-a3c5-f301ca0faaf9 started at 2023-03-16 14:50:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container alpine-nnp-true-39d13e6b-e843-403a-a3c5-f301ca0faaf9 ready: false, restart count 0 Mar 16 14:50:15.799: INFO: simpletest.rc-w8vhz started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:15.799: INFO: ss2-0 started at 2023-03-16 14:50:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container webserver ready: false, restart count 0 Mar 16 14:50:15.799: INFO: netserver-1 started at 2023-03-16 14:49:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.799: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:15.799: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:15.799: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:15.799: INFO: pod1 started at 2023-03-16 14:48:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:15.799: INFO: netserver-1 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.799: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:15.799: INFO: execpod5nlxl started at 2023-03-16 14:50:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:15.799: INFO: simpletest.rc-zh8lq started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:15.799: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:15.799: INFO: netserver-1 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:15.799: INFO: termination-message-container75463329-156c-4aee-90e1-d7742fa87088 started at 2023-03-16 14:50:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container termination-message-container ready: false, restart count 0 Mar 16 14:50:15.799: INFO: externalname-service-tzjnf started at 2023-03-16 14:50:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:15.799: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:50:16.297: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:16.297: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:50:16.356: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 4445 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:49:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:16.356: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:50:16.428: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:50:16.510: INFO: ss2-2 started at 2023-03-16 14:49:53 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:16.510: INFO: affinity-nodeport-fqkx9 started at 2023-03-16 14:48:54 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container affinity-nodeport ready: true, restart count 0 Mar 16 14:50:16.510: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:16.510: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:50:16.510: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container coredns ready: true, restart count 0 Mar 16 14:50:16.510: INFO: netserver-2 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:16.510: INFO: externalname-service-zfqv8 started at 2023-03-16 14:50:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:50:16.510: INFO: execpodktw4g started at 2023-03-16 14:49:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:16.510: INFO: up-down-1-9v72k started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container up-down-1 ready: true, restart count 0 Mar 16 14:50:16.510: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:16.510: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:16.510: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:16.510: INFO: simpletest.rc-zcnrt started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:16.510: INFO: sample-webhook-deployment-7497495989-bwn2l started at 2023-03-16 14:50:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container sample-webhook ready: false, restart count 0 Mar 16 14:50:16.510: INFO: netserver-2 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:16.510: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:16.510: INFO: kube-dns-autoscaler-7b444c59c9-2q9j6 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:16.510: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:50:16.982: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:50:16.982: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:50:17.099: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 4637 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2023-03-16 14:49:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:17.100: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:50:17.189: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:50:17.366: INFO: pod-with-prestop-http-hook started at 2023-03-16 14:50:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container pod-with-prestop-http-hook ready: true, restart count 0 Mar 16 14:50:17.366: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:17.366: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:17.366: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:17.366: INFO: affinity-nodeport-fx2r5 started at 2023-03-16 14:48:54 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container affinity-nodeport ready: true, restart count 0 Mar 16 14:50:17.366: INFO: pod-handle-http-request started at 2023-03-16 14:50:11 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:17.366: INFO: Container container-handle-http-request ready: true, restart count 0 Mar 16 14:50:17.366: INFO: Container container-handle-https-request ready: true, restart count 0 Mar 16 14:50:17.366: INFO: busybox-a7cb690b-9c7f-48cb-a325-bb02d85b059f started at 2023-03-16 14:49:32 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container busybox ready: true, restart count 0 Mar 16 14:50:17.366: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:17.366: INFO: pod-logs-websocket-779daf25-2d8a-41e5-bb17-0350eedb1d26 started at 2023-03-16 14:50:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container main ready: false, restart count 0 Mar 16 14:50:17.366: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:17.366: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:50:17.366: INFO: pod-0 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container donothing ready: true, restart count 0 Mar 16 14:50:17.366: INFO: pod2 started at 2023-03-16 14:49:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:17.366: INFO: test-runtimeclass-runtimeclass-1868-preconfigured-handler-xj4t4 started at 2023-03-16 14:50:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container test ready: false, restart count 0 Mar 16 14:50:17.366: INFO: up-down-1-vsp2v started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container up-down-1 ready: true, restart count 0 Mar 16 14:50:17.366: INFO: netserver-3 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:17.366: INFO: netserver-3 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:17.366: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:17.811: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:50:17.811 (3.682s) < Exit [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - dump namespaces | framework.go:209 @ 03/16/23 14:50:17.811 (3.683s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - tear down framework | framework.go:206 @ 03/16/23 14:50:17.811 STEP: Destroying namespace "port-forwarding-6874" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:50:17.811 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl Port forwarding - tear down framework | framework.go:206 @ 03/16/23 14:50:17.94 (130ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:50:17.941 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:50:17.941 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\sreturn\scommand\sexit\scodes\sexecing\sinto\sa\scontainer\swith\sa\sfailing\scommand$'
[FAILED] Expected <int>: 1 to equal <int>: 42 In [It] at: test/e2e/kubectl/kubectl.go:547 @ 03/16/23 14:50:09.257from junit_01.xml
> Enter [BeforeEach] [sig-cli] Kubectl client - set up framework | framework.go:191 @ 03/16/23 14:49:25.48 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:49:25.48 Mar 16 14:49:25.480: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename kubectl - test/e2e/framework/framework.go:250 @ 03/16/23 14:49:25.482 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:49:25.623 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:49:25.705 < Exit [BeforeEach] [sig-cli] Kubectl client - set up framework | framework.go:191 @ 03/16/23 14:49:25.785 (305ms) > Enter [BeforeEach] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:49:25.785 < Exit [BeforeEach] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:49:25.785 (0s) > Enter [BeforeEach] [sig-cli] Kubectl client - test/e2e/kubectl/kubectl.go:276 @ 03/16/23 14:49:25.785 < Exit [BeforeEach] [sig-cli] Kubectl client - test/e2e/kubectl/kubectl.go:276 @ 03/16/23 14:49:25.785 (0s) > Enter [BeforeEach] Simple pod - test/e2e/kubectl/kubectl.go:413 @ 03/16/23 14:49:25.785 STEP: creating the pod from - test/e2e/kubectl/kubectl.go:414 @ 03/16/23 14:49:25.785 Mar 16 14:49:25.785: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-5986 create -f -' Mar 16 14:49:26.444: INFO: stderr: "" Mar 16 14:49:26.444: INFO: stdout: "pod/httpd created\n" < Exit [BeforeEach] Simple pod - test/e2e/kubectl/kubectl.go:413 @ 03/16/23 14:49:38.912 (13.127s) > Enter [It] execing into a container with a failing command - test/e2e/kubectl/kubectl.go:541 @ 03/16/23 14:49:38.912 Mar 16 14:49:38.912: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-5986 exec httpd --pod-running-timeout=2m0s -- /bin/sh -c exit 42' Mar 16 14:50:09.257: INFO: rc: 1 [FAILED] Expected <int>: 1 to equal <int>: 42 In [It] at: test/e2e/kubectl/kubectl.go:547 @ 03/16/23 14:50:09.257 < Exit [It] execing into a container with a failing command - test/e2e/kubectl/kubectl.go:541 @ 03/16/23 14:50:09.257 (30.345s) > Enter [AfterEach] Simple pod - test/e2e/kubectl/kubectl.go:419 @ 03/16/23 14:50:09.257 STEP: using delete to clean up resources - test/e2e/kubectl/kubectl.go:200 @ 03/16/23 14:50:09.257 Mar 16 14:50:09.257: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-5986 delete --grace-period=0 --force -f -' Mar 16 14:50:09.574: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Mar 16 14:50:09.574: INFO: stdout: "pod \"httpd\" force deleted\n" Mar 16 14:50:09.574: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-5986 get rc,svc -l name=httpd --no-headers' Mar 16 14:50:09.929: INFO: stderr: "No resources found in kubectl-5986 namespace.\n" Mar 16 14:50:09.929: INFO: stdout: "" Mar 16 14:50:09.929: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-5986 get pods -l name=httpd -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Mar 16 14:50:10.131: INFO: stderr: "" Mar 16 14:50:10.131: INFO: stdout: "" < Exit [AfterEach] Simple pod - test/e2e/kubectl/kubectl.go:419 @ 03/16/23 14:50:10.131 (873ms) > Enter [AfterEach] [sig-cli] Kubectl client - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:50:10.131 Mar 16 14:50:10.131: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-cli] Kubectl client - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:50:10.178 (47ms) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:50:10.178 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:50:10.178 (0s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - dump namespaces | framework.go:209 @ 03/16/23 14:50:10.178 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:50:10.178 STEP: Collecting events from namespace "kubectl-5986". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:50:10.178 STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:50:10.222 Mar 16 14:50:10.222: INFO: At 2023-03-16 14:49:26 +0000 UTC - event for httpd: {default-scheduler } Scheduled: Successfully assigned kubectl-5986/httpd to bootstrap-e2e-minion-group-z4ns Mar 16 14:50:10.222: INFO: At 2023-03-16 14:49:28 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine Mar 16 14:50:10.222: INFO: At 2023-03-16 14:49:28 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container httpd Mar 16 14:50:10.222: INFO: At 2023-03-16 14:49:28 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container httpd Mar 16 14:50:10.222: INFO: At 2023-03-16 14:50:09 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Killing: Stopping container httpd Mar 16 14:50:10.264: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:50:10.264: INFO: Mar 16 14:50:10.413: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:50:10.483: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 660 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:47:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:10.484: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:50:10.558: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:50:10.658: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:10.658: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:10.658: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:10.658: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:50:10.658: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:50:10.658: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:50:10.658: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:50:10.658: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:50:10.658: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:50:10.658: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:50:10.658: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:10.658: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:50:10.901: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:50:10.901: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:10.973: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 4261 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:49:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:59 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:10.973: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:11.046: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:11.182: INFO: sample-webhook-deployment-7497495989-n2jht started at 2023-03-16 14:50:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container sample-webhook ready: true, restart count 0 Mar 16 14:50:11.182: INFO: security-context-e448a778-33f8-4f10-b846-486c72d064d3 started at 2023-03-16 14:50:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container test-container ready: false, restart count 0 Mar 16 14:50:11.182: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:11.182: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:11.182: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-n2n4h started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-z995c started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-qcgr5 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: pfpod started at 2023-03-16 14:49:35 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:11.182: INFO: Container portforwardtester ready: true, restart count 0 Mar 16 14:50:11.182: INFO: Container readiness ready: true, restart count 0 Mar 16 14:50:11.182: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:11.182: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:50:11.182: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-lkbbq started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: execpod-affinityp5ndd started at 2023-03-16 14:49:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:11.182: INFO: host-test-container-pod started at 2023-03-16 14:49:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-nvpbj started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: ss2-1 started at 2023-03-16 14:50:03 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-hxttt started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-gtbnw started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:11.182: INFO: up-down-1-bs2xk started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container up-down-1 ready: true, restart count 0 Mar 16 14:50:11.182: INFO: test-container-pod started at 2023-03-16 14:50:04 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-kzc2l started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-klcwm started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-stlpj started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: dns-test-0667a486-49b4-4b08-9ca9-fd90309002a5 started at 2023-03-16 14:50:00 +0000 UTC (0+3 container statuses recorded) Mar 16 14:50:11.182: INFO: Container jessie-querier ready: true, restart count 0 Mar 16 14:50:11.182: INFO: Container querier ready: true, restart count 0 Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: pod-2 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container donothing ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-pkp62 started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-s4b9c started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:50:11.182: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container httpd ready: true, restart count 0 Mar 16 14:50:11.182: INFO: netserver-0 started at 2023-03-16 14:49:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: verify-service-up-exec-pod-b5kdb started at 2023-03-16 14:49:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:11.182: INFO: busybox-a5a35a8e-81ce-4aed-8ded-436003287c63 started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container busybox ready: true, restart count 0 Mar 16 14:50:11.182: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:49:53 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:11.182: INFO: startup-6a3107dc-925f-44e7-a222-640c4b0abd9e started at 2023-03-16 14:49:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container busybox ready: true, restart count 0 Mar 16 14:50:11.182: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:50:11.182: INFO: test-webserver-ffa602b2-b01a-48fb-94e6-32dd96b9bc8e started at 2023-03-16 14:49:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container test-webserver ready: false, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-n72sr started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: netserver-0 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:11.182: INFO: netserver-0 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-w2k9t started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.182: INFO: test-container-pod started at 2023-03-16 14:49:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.182: INFO: pod-1 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container donothing ready: true, restart count 0 Mar 16 14:50:11.182: INFO: simpletest.rc-h2llm started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.182: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.623: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:50:11.623: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:11.673: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 4339 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:11.674: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:11.728: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:11.845: INFO: simpletest.rc-w8vhz started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: netserver-1 started at 2023-03-16 14:49:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.846: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:11.846: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:11.846: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-vfj5k started at 2023-03-16 14:48:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: pod1 started at 2023-03-16 14:48:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:11.846: INFO: netserver-1 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.846: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-r9d89 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-zh8lq started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-kr2lv started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-m87jj started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: netserver-1 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:11.846: INFO: externalname-service-tzjnf started at 2023-03-16 14:50:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-t2m64 started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: affinity-nodeport-88b8w started at 2023-03-16 14:48:54 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container affinity-nodeport ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-s2g5g started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-rzzhq started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-q5l7d started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container coredns ready: true, restart count 0 Mar 16 14:50:11.846: INFO: pod-server-2 started at 2023-03-16 14:50:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-kh8dm started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-mgtpp started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:11.846: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:50:11.846: INFO: simpletest.rc-mwjrv started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:11.846: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.183: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:50:12.183: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:50:12.320: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 4445 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:49:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:12.320: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:50:12.441: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:50:12.670: INFO: simpletest.rc-wd6tn started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-pkf25 started at 2023-03-16 14:48:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-nlq5m started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-qjfn7 started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:12.670: INFO: netserver-2 started at 2023-03-16 14:49:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-m59rp started at 2023-03-16 14:49:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: externalname-service-zfqv8 started at 2023-03-16 14:50:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:50:12.670: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:50:12.670: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container coredns ready: true, restart count 0 Mar 16 14:50:12.670: INFO: netserver-2 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:12.670: INFO: execpodktw4g started at 2023-03-16 14:49:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:12.670: INFO: up-down-1-9v72k started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container up-down-1 ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-zcnrt started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:12.670: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:12.670: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-z9wm5 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-t9kc9 started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: netserver-2 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:12.670: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:12.670: INFO: kube-dns-autoscaler-7b444c59c9-2q9j6 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:50:12.670: INFO: ss2-2 started at 2023-03-16 14:49:53 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-tg98d started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-tmbt7 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-tdwc8 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:12.670: INFO: affinity-nodeport-fqkx9 started at 2023-03-16 14:48:54 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container affinity-nodeport ready: true, restart count 0 Mar 16 14:50:12.670: INFO: simpletest.rc-xm9ll started at 2023-03-16 14:48:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:12.670: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.260: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:50:13.260: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:50:13.370: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 4637 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2023-03-16 14:49:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:49:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:50:13.370: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:50:13.482: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:50:13.611: INFO: ss2-0 started at 2023-03-16 14:49:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container webserver ready: false, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-zl8dn started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: netserver-3 started at 2023-03-16 14:49:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:13.611: INFO: netserver-3 started at 2023-03-16 14:49:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:13.611: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:13.611: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:50:13.611: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:50:13.611: INFO: affinity-nodeport-fx2r5 started at 2023-03-16 14:48:54 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container affinity-nodeport ready: true, restart count 0 Mar 16 14:50:13.611: INFO: netserver-3 started at 2023-03-16 14:49:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container webserver ready: true, restart count 0 Mar 16 14:50:13.611: INFO: pod-handle-http-request started at 2023-03-16 14:50:11 +0000 UTC (0+2 container statuses recorded) Mar 16 14:50:13.611: INFO: Container container-handle-http-request ready: false, restart count 0 Mar 16 14:50:13.611: INFO: Container container-handle-https-request ready: false, restart count 0 Mar 16 14:50:13.611: INFO: busybox-a7cb690b-9c7f-48cb-a325-bb02d85b059f started at 2023-03-16 14:49:32 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container busybox ready: true, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-twqzc started at 2023-03-16 14:48:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-tbdj8 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-prnxm started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-x8nl6 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:50:13.611: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:50:13.611: INFO: pod-logs-websocket-779daf25-2d8a-41e5-bb17-0350eedb1d26 started at <nil> (0+0 container statuses recorded) Mar 16 14:50:13.611: INFO: simpletest.rc-qh2fh started at 2023-03-16 14:48:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: pod2 started at 2023-03-16 14:49:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:50:13.611: INFO: test-runtimeclass-runtimeclass-1868-preconfigured-handler-xj4t4 started at 2023-03-16 14:50:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container test ready: false, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-mvr9n started at 2023-03-16 14:48:56 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: pod-terminate-status-2-14 started at 2023-03-16 14:50:11 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container fail ready: false, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-sm8hh started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: pod-0 started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container donothing ready: true, restart count 0 Mar 16 14:50:13.611: INFO: simpletest.rc-wlvbp started at 2023-03-16 14:48:58 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container nginx ready: true, restart count 0 Mar 16 14:50:13.611: INFO: up-down-1-vsp2v started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:50:13.611: INFO: Container up-down-1 ready: true, restart count 0 Mar 16 14:50:16.149: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:50:16.149 (5.97s) < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - dump namespaces | framework.go:209 @ 03/16/23 14:50:16.149 (5.971s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - tear down framework | framework.go:206 @ 03/16/23 14:50:16.149 STEP: Destroying namespace "kubectl-5986" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:50:16.149 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - tear down framework | framework.go:206 @ 03/16/23 14:50:16.206 (57ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:50:16.206 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:50:16.206 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\ssupport\sexec$'
[FAILED] error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 exec --pod-running-timeout=2m0s httpd -- echo running in container: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 In [It] at: test/e2e/framework/kubectl/builder.go:87 @ 03/16/23 14:53:29.786from junit_01.xml
> Enter [BeforeEach] [sig-cli] Kubectl client - set up framework | framework.go:191 @ 03/16/23 14:52:45.761 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:52:45.761 Mar 16 14:52:45.761: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename kubectl - test/e2e/framework/framework.go:250 @ 03/16/23 14:52:45.762 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:52:45.891 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:52:46.018 < Exit [BeforeEach] [sig-cli] Kubectl client - set up framework | framework.go:191 @ 03/16/23 14:52:46.109 (348ms) > Enter [BeforeEach] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:52:46.109 < Exit [BeforeEach] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:52:46.109 (0s) > Enter [BeforeEach] [sig-cli] Kubectl client - test/e2e/kubectl/kubectl.go:276 @ 03/16/23 14:52:46.109 < Exit [BeforeEach] [sig-cli] Kubectl client - test/e2e/kubectl/kubectl.go:276 @ 03/16/23 14:52:46.109 (0s) > Enter [BeforeEach] Simple pod - test/e2e/kubectl/kubectl.go:413 @ 03/16/23 14:52:46.109 STEP: creating the pod from apiVersion: v1 kind: Pod metadata: name: httpd labels: name: httpd spec: containers: - name: httpd image: registry.k8s.io/e2e-test-images/httpd:2.4.38-4 ports: - containerPort: 80 readinessProbe: httpGet: path: / port: 80 initialDelaySeconds: 5 timeoutSeconds: 5 - test/e2e/kubectl/kubectl.go:414 @ 03/16/23 14:52:46.109 Mar 16 14:52:46.109: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 create -f -' Mar 16 14:52:46.784: INFO: stderr: "" Mar 16 14:52:46.784: INFO: stdout: "pod/httpd created\n" < Exit [BeforeEach] Simple pod - test/e2e/kubectl/kubectl.go:413 @ 03/16/23 14:52:59.397 (13.288s) > Enter [It] should support exec - test/e2e/kubectl/kubectl.go:423 @ 03/16/23 14:52:59.397 STEP: executing a command in the container - test/e2e/kubectl/kubectl.go:424 @ 03/16/23 14:52:59.397 Mar 16 14:52:59.397: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 exec --pod-running-timeout=2m0s httpd -- echo running in container' Mar 16 14:53:29.785: INFO: rc: 1 Mar 16 14:53:29.786: INFO: Unexpected error: <exec.CodeExitError>: error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 exec --pod-running-timeout=2m0s httpd -- echo running in container: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 { Err: <*errors.errorString | 0xc0016a9690>{ s: "error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 exec --pod-running-timeout=2m0s httpd -- echo running in container:\nCommand stdout:\n\nstderr:\nError from server: error dialing backend: dial timeout, backstop\n\nerror:\nexit status 1", }, Code: 1, } [FAILED] error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 exec --pod-running-timeout=2m0s httpd -- echo running in container: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 In [It] at: test/e2e/framework/kubectl/builder.go:87 @ 03/16/23 14:53:29.786 < Exit [It] should support exec - test/e2e/kubectl/kubectl.go:423 @ 03/16/23 14:53:29.786 (30.389s) > Enter [AfterEach] Simple pod - test/e2e/kubectl/kubectl.go:419 @ 03/16/23 14:53:29.786 STEP: using delete to clean up resources - test/e2e/kubectl/kubectl.go:200 @ 03/16/23 14:53:29.786 Mar 16 14:53:29.786: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 delete --grace-period=0 --force -f -' Mar 16 14:53:30.078: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Mar 16 14:53:30.078: INFO: stdout: "pod \"httpd\" force deleted\n" Mar 16 14:53:30.079: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 get rc,svc -l name=httpd --no-headers' Mar 16 14:53:30.321: INFO: stderr: "No resources found in kubectl-364 namespace.\n" Mar 16 14:53:30.321: INFO: stdout: "" Mar 16 14:53:30.321: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-364 get pods -l name=httpd -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Mar 16 14:53:30.520: INFO: stderr: "" Mar 16 14:53:30.520: INFO: stdout: "" < Exit [AfterEach] Simple pod - test/e2e/kubectl/kubectl.go:419 @ 03/16/23 14:53:30.52 (734ms) > Enter [AfterEach] [sig-cli] Kubectl client - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:53:30.52 Mar 16 14:53:30.520: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-cli] Kubectl client - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:53:30.568 (49ms) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:53:30.568 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:53:30.568 (0s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - dump namespaces | framework.go:209 @ 03/16/23 14:53:30.568 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:53:30.568 STEP: Collecting events from namespace "kubectl-364". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:53:30.569 STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:53:30.643 Mar 16 14:53:30.643: INFO: At 2023-03-16 14:52:46 +0000 UTC - event for httpd: {default-scheduler } Scheduled: Successfully assigned kubectl-364/httpd to bootstrap-e2e-minion-group-z4ns Mar 16 14:53:30.643: INFO: At 2023-03-16 14:52:49 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine Mar 16 14:53:30.643: INFO: At 2023-03-16 14:52:49 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container httpd Mar 16 14:53:30.643: INFO: At 2023-03-16 14:52:49 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container httpd Mar 16 14:53:30.643: INFO: At 2023-03-16 14:53:29 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-z4ns} Killing: Stopping container httpd Mar 16 14:53:30.684: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:53:30.684: INFO: Mar 16 14:53:30.732: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:53:30.774: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 12046 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:52:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:30.774: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:53:30.817: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:53:30.866: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:53:30.866: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:53:30.866: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:53:30.866: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:53:30.866: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:53:30.866: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:53:30.866: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:53:30.866: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:30.866: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:53:30.866: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:30.866: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:30.866: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:31.096: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:53:31.096: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:31.138: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 8265 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:31.138: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:31.186: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:31.276: INFO: netserver-0 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container webserver ready: false, restart count 0 Mar 16 14:53:31.276: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:31.276: INFO: pause-pod-1 started at 2023-03-16 14:53:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:31.276: INFO: pod-handle-http-request started at 2023-03-16 14:51:36 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:31.276: INFO: Container container-handle-8090-request ready: true, restart count 0 Mar 16 14:53:31.276: INFO: Container container-handle-9090-request ready: true, restart count 0 Mar 16 14:53:31.276: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:53:31.276: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container httpd ready: true, restart count 0 Mar 16 14:53:31.276: INFO: busybox-a5a35a8e-81ce-4aed-8ded-436003287c63 started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container busybox ready: true, restart count 0 Mar 16 14:53:31.276: INFO: to-be-attached-pod started at 2023-03-16 14:53:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container container1 ready: true, restart count 0 Mar 16 14:53:31.276: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.276: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:53:31.276: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:31.276: INFO: netserver-0 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.276: INFO: test-rolling-update-with-lb-66756c6d5c-zbc84 started at 2023-03-16 14:53:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:31.276: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:31.276: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:31.276: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:31.276: INFO: coredns-8f5847b64-z5pl6 started at 2023-03-16 14:53:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:31.276: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:31.276: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:53:31.276: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:53:31.276: INFO: execpod7b67j started at 2023-03-16 14:53:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:31.276: INFO: netserver-0 started at 2023-03-16 14:52:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.276: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.276: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:53:31.276: INFO: httpd started at 2023-03-16 14:53:30 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.276: INFO: Container httpd ready: false, restart count 0 Mar 16 14:53:31.510: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:31.510: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:31.555: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:31.556: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:31.603: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:31.678: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.678: INFO: affinity-clusterip-transition-7prln started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:53:31.678: INFO: pause-pod-0 started at 2023-03-16 14:51:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:31.678: INFO: netserver-1 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.678: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:31.678: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.678: INFO: execpodnsv2z started at 2023-03-16 14:52:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:31.678: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.678: INFO: pod1 started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:31.678: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:53:31.678: INFO: pod1 started at 2023-03-16 14:52:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:31.678: INFO: test-rolling-update-with-lb-66756c6d5c-9l8fw started at 2023-03-16 14:53:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:31.678: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:31.678: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:31.678: INFO: netserver-1 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container webserver ready: false, restart count 0 Mar 16 14:53:31.678: INFO: nodeport-update-service-czr4d started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:53:31.678: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:31.678: INFO: rs-6kdwb started at 2023-03-16 14:52:35 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container donothing ready: false, restart count 0 Mar 16 14:53:31.678: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:31.678: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:31.678: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:31.678: INFO: execpodf4mm6 started at 2023-03-16 14:51:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:31.678: INFO: pod-init-3cacd6ff-947a-479c-b842-ff17d3d0438c started at 2023-03-16 14:53:22 +0000 UTC (2+1 container statuses recorded) Mar 16 14:53:31.678: INFO: Init container init1 ready: true, restart count 0 Mar 16 14:53:31.678: INFO: Init container init2 ready: true, restart count 0 Mar 16 14:53:31.678: INFO: Container run1 ready: true, restart count 0 Mar 16 14:53:31.883: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:31.883: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:53:31.926: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 6652 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:31.926: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:53:31.970: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:53:32.029: INFO: host-test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.029: INFO: sample-webhook-deployment-7497495989-9md9b started at 2023-03-16 14:53:23 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container sample-webhook ready: true, restart count 0 Mar 16 14:53:32.029: INFO: kube-dns-autoscaler-7b444c59c9-2q9j6 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:53:32.029: INFO: nodeport-test-dg97z started at 2023-03-16 14:53:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container nodeport-test ready: true, restart count 0 Mar 16 14:53:32.029: INFO: pod2 started at 2023-03-16 14:52:33 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.029: INFO: execpod-affinityztffd started at 2023-03-16 14:53:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.029: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:32.029: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:32.029: INFO: affinity-clusterip-transition-28fjr started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:53:32.029: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.029: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:53:32.029: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.029: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:52:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.029: INFO: netserver-2 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.029: INFO: test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.029: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:32.029: INFO: simpletest.deployment-5d9bdff859-jtc62 started at 2023-03-16 14:53:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container nginx ready: true, restart count 0 Mar 16 14:53:32.029: INFO: busybox-user-65534-1ec80362-2c33-466d-bafd-59571ecd7084 started at 2023-03-16 14:53:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container busybox-user-65534-1ec80362-2c33-466d-bafd-59571ecd7084 ready: false, restart count 0 Mar 16 14:53:32.029: INFO: e2e-net-exec started at 2023-03-16 14:53:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 16 14:53:32.029: INFO: verify-service-up-exec-pod-r45vt started at 2023-03-16 14:52:22 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.029: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.029: INFO: test-grpc-78fcda4d-0543-4b7d-84c7-d1b98379ae7a started at 2023-03-16 14:52:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:32.029: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.029: INFO: netserver-2 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.029: INFO: Container webserver ready: false, restart count 0 Mar 16 14:53:32.029: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:32.029: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:32.029: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:32.256: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:53:32.256: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:53:32.307: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 11234 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:52:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:32.307: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:53:32.353: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:53:32.428: INFO: ephemeral-containers-target-pod started at 2023-03-16 14:53:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container test-container-1 ready: true, restart count 0 Mar 16 14:53:32.428: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.428: INFO: simpletest.deployment-5d9bdff859-s8xln started at 2023-03-16 14:53:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container nginx ready: true, restart count 0 Mar 16 14:53:32.428: INFO: test-rolling-update-with-lb-66756c6d5c-jfk8b started at 2023-03-16 14:53:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:32.428: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:32.428: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:32.428: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:32.428: INFO: nodeport-test-zzq4p started at 2023-03-16 14:53:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container nodeport-test ready: true, restart count 0 Mar 16 14:53:32.428: INFO: affinity-clusterip-transition-p88jm started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:53:32.428: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.428: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:32.428: INFO: execpod-noendpoints2fwr6 started at 2023-03-16 14:53:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.428: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:32.428: INFO: execpodz4xt8 started at 2023-03-16 14:53:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.428: INFO: e2e-net-server started at 2023-03-16 14:53:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container e2e-net-server ready: false, restart count 0 Mar 16 14:53:32.428: INFO: netserver-3 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container webserver ready: false, restart count 0 Mar 16 14:53:32.428: INFO: test-container-pod started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.428: INFO: nodeport-update-service-d27z2 started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:53:32.428: INFO: webserver-pod started at 2023-03-16 14:53:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:32.428: INFO: coredns-8f5847b64-492qq started at 2023-03-16 14:53:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:32.428: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.428: INFO: test-grpc-31967f11-8259-4e30-a7c5-d8b1d0d6ad49 started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:32.428: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:53:32.428: INFO: rand-non-local-b8ptb started at 2023-03-16 14:53:22 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container c ready: false, restart count 0 Mar 16 14:53:32.428: INFO: netserver-3 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:32.428: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:32.646: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:53:32.646 (2.077s) < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - dump namespaces | framework.go:209 @ 03/16/23 14:53:32.646 (2.077s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - tear down framework | framework.go:206 @ 03/16/23 14:53:32.646 STEP: Destroying namespace "kubectl-364" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:53:32.646 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - tear down framework | framework.go:206 @ 03/16/23 14:53:32.691 (45ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:53:32.691 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:53:32.691 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-cli\]\sKubectl\sclient\sSimple\spod\sshould\ssupport\sexec\sthrough\san\sHTTP\sproxy$'
[FAILED] error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 exec --pod-running-timeout=2m0s httpd -- echo running in container: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 In [It] at: test/e2e/framework/kubectl/builder.go:87 @ 03/16/23 14:54:32.196from junit_01.xml
> Enter [BeforeEach] [sig-cli] Kubectl client - set up framework | framework.go:191 @ 03/16/23 14:53:47.207 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:53:47.207 Mar 16 14:53:47.207: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename kubectl - test/e2e/framework/framework.go:250 @ 03/16/23 14:53:47.208 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:53:47.337 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:53:47.42 < Exit [BeforeEach] [sig-cli] Kubectl client - set up framework | framework.go:191 @ 03/16/23 14:53:47.501 (294ms) > Enter [BeforeEach] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:53:47.501 < Exit [BeforeEach] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:53:47.501 (0s) > Enter [BeforeEach] [sig-cli] Kubectl client - test/e2e/kubectl/kubectl.go:276 @ 03/16/23 14:53:47.501 < Exit [BeforeEach] [sig-cli] Kubectl client - test/e2e/kubectl/kubectl.go:276 @ 03/16/23 14:53:47.501 (0s) > Enter [BeforeEach] Simple pod - test/e2e/kubectl/kubectl.go:413 @ 03/16/23 14:53:47.501 STEP: creating the pod from - test/e2e/kubectl/kubectl.go:414 @ 03/16/23 14:53:47.501 Mar 16 14:53:47.502: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 create -f -' Mar 16 14:53:48.337: INFO: stderr: "" Mar 16 14:53:48.337: INFO: stdout: "pod/httpd created\n" < Exit [BeforeEach] Simple pod - test/e2e/kubectl/kubectl.go:413 @ 03/16/23 14:54:00.87 (13.368s) > Enter [It] should support exec through an HTTP proxy - test/e2e/kubectl/kubectl.go:471 @ 03/16/23 14:54:00.87 STEP: Starting http_proxy - test/e2e/kubectl/kubectl.go:477 @ 03/16/23 14:54:00.87 STEP: Running kubectl via an HTTP proxy using https_proxy - test/e2e/kubectl/kubectl.go:488 @ 03/16/23 14:54:00.87 Mar 16 14:54:00.870: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 exec --pod-running-timeout=2m0s httpd -- echo running in container' Proxy Request: CONNECT 34.105.125.69:443 HTTP/1.1 Host: 34.105.125.69:443 User-Agent: Go-http-client/1.1 Proxy Request: CONNECT 34.105.125.69:443 HTTP/1.1 Host: 34.105.125.69:443 User-Agent: Go-http-client/1.1 Server read close, host=34.105.125.69:443 Server write close, host=34.105.125.69:443 Done handling CONNECT request, host=34.105.125.69:443 Server write close, host=34.105.125.69:443 Mar 16 14:54:01.864: INFO: stderr: "" Mar 16 14:54:01.864: INFO: stdout: "running in container\n" STEP: Running kubectl via an HTTP proxy using HTTPS_PROXY - test/e2e/kubectl/kubectl.go:488 @ 03/16/23 14:54:01.864 Mar 16 14:54:01.864: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 exec --pod-running-timeout=2m0s httpd -- echo running in container' Proxy Request: CONNECT 34.105.125.69:443 HTTP/1.1 Host: 34.105.125.69:443 User-Agent: Go-http-client/1.1 Proxy Request: CONNECT 34.105.125.69:443 HTTP/1.1 Host: 34.105.125.69:443 User-Agent: Go-http-client/1.1 Server write close, host=34.105.125.69:443 Server write close, host=34.105.125.69:443 Mar 16 14:54:32.195: INFO: rc: 1 Mar 16 14:54:32.195: INFO: Unexpected error: <exec.CodeExitError>: error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 exec --pod-running-timeout=2m0s httpd -- echo running in container: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 { Err: <*errors.errorString | 0xc00011b7d0>{ s: "error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 exec --pod-running-timeout=2m0s httpd -- echo running in container:\nCommand stdout:\n\nstderr:\nError from server: error dialing backend: dial timeout, backstop\n\nerror:\nexit status 1", }, Code: 1, } [FAILED] error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 exec --pod-running-timeout=2m0s httpd -- echo running in container: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 In [It] at: test/e2e/framework/kubectl/builder.go:87 @ 03/16/23 14:54:32.196 < Exit [It] should support exec through an HTTP proxy - test/e2e/kubectl/kubectl.go:471 @ 03/16/23 14:54:32.196 (31.326s) > Enter [AfterEach] Simple pod - test/e2e/kubectl/kubectl.go:419 @ 03/16/23 14:54:32.196 STEP: using delete to clean up resources - test/e2e/kubectl/kubectl.go:200 @ 03/16/23 14:54:32.196 Mar 16 14:54:32.196: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 delete --grace-period=0 --force -f -' Mar 16 14:54:32.451: INFO: stderr: "Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.\n" Mar 16 14:54:32.451: INFO: stdout: "pod \"httpd\" force deleted\n" Mar 16 14:54:32.452: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 get rc,svc -l name=httpd --no-headers' Mar 16 14:54:32.703: INFO: stderr: "No resources found in kubectl-3492 namespace.\n" Mar 16 14:54:32.703: INFO: stdout: "" Mar 16 14:54:32.703: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=kubectl-3492 get pods -l name=httpd -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}' Mar 16 14:54:32.903: INFO: stderr: "" Mar 16 14:54:32.903: INFO: stdout: "" < Exit [AfterEach] Simple pod - test/e2e/kubectl/kubectl.go:419 @ 03/16/23 14:54:32.903 (707ms) > Enter [AfterEach] [sig-cli] Kubectl client - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:54:32.903 Mar 16 14:54:32.903: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-cli] Kubectl client - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:54:32.947 (44ms) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:54:32.947 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:54:32.947 (0s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - dump namespaces | framework.go:209 @ 03/16/23 14:54:32.947 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:54:32.947 STEP: Collecting events from namespace "kubectl-3492". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:54:32.947 STEP: Found 5 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:54:33.071 Mar 16 14:54:33.071: INFO: At 2023-03-16 14:53:48 +0000 UTC - event for httpd: {default-scheduler } Scheduled: Successfully assigned kubectl-3492/httpd to bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:33.071: INFO: At 2023-03-16 14:53:50 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/httpd:2.4.38-4" already present on machine Mar 16 14:54:33.071: INFO: At 2023-03-16 14:53:50 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container httpd Mar 16 14:54:33.071: INFO: At 2023-03-16 14:53:51 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container httpd Mar 16 14:54:33.071: INFO: At 2023-03-16 14:54:32 +0000 UTC - event for httpd: {kubelet bootstrap-e2e-minion-group-hzx6} Killing: Stopping container httpd Mar 16 14:54:33.113: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:54:33.113: INFO: Mar 16 14:54:33.156: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:54:33.315: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 12046 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:52:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:33.316: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:54:33.472: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:54:33.587: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:54:33.587: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:54:33.587: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:54:33.587: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:54:33.587: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:54:33.587: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:54:33.587: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:33.587: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:33.587: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:33.587: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:54:33.587: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:33.587: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:54:33.861: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:54:33.861: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:33.907: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 16125 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:33.907: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:33.954: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:34.037: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:34.037: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:34.037: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:34.037: INFO: netserver-0 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:34.037: INFO: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac ready: true, restart count 0 Mar 16 14:54:34.037: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:34.037: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:34.037: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:54:34.037: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:54:34.037: INFO: execpod7b67j started at 2023-03-16 14:53:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:34.037: INFO: netserver-0 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.037: INFO: netserver-0 started at 2023-03-16 14:52:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.037: INFO: all-pods-removed-tv8rl started at 2023-03-16 14:54:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container c ready: true, restart count 0 Mar 16 14:54:34.037: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:34.037: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.037: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:34.037: INFO: netserver-0 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.037: INFO: netserver-0 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.037: INFO: all-pods-removed-z7m42 started at 2023-03-16 14:54:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container c ready: true, restart count 0 Mar 16 14:54:34.037: INFO: execpod-affinityn8trg started at 2023-03-16 14:54:18 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:34.037: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:54:34.037: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container httpd ready: true, restart count 0 Mar 16 14:54:34.037: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.037: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:54:34.037: INFO: terminate-cmd-rpn2679f480-1f82-4cf4-8676-c3d977da3660 started at 2023-03-16 14:54:30 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container terminate-cmd-rpn ready: false, restart count 0 Mar 16 14:54:34.037: INFO: affinity-nodeport-transition-g77pz started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 16 14:54:34.037: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:34.037: INFO: up-down-3-6hgmw started at 2023-03-16 14:54:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.037: INFO: Container up-down-3 ready: true, restart count 0 Mar 16 14:54:34.399: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:34.399: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:34.449: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:34.449: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:34.497: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:34.569: INFO: affinity-clusterip-transition-7prln started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:34.569: INFO: sample-crd-conversion-webhook-deployment-5969648595-nx5tn started at 2023-03-16 14:54:30 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container sample-crd-conversion-webhook ready: true, restart count 0 Mar 16 14:54:34.569: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container coredns ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: test-container-pod started at 2023-03-16 14:54:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:34.569: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:34.569: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: up-down-3-zhgsc started at 2023-03-16 14:54:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container up-down-3 ready: true, restart count 0 Mar 16 14:54:34.569: INFO: pod-client started at 2023-03-16 14:54:19 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container pod-client ready: true, restart count 0 Mar 16 14:54:34.569: INFO: test-ss-0 started at 2023-03-16 14:54:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: nodeport-update-service-czr4d started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: rs-6kdwb started at 2023-03-16 14:52:35 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container donothing ready: false, restart count 0 Mar 16 14:54:34.569: INFO: exceed-active-deadline-5d7fv started at 2023-03-16 14:54:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container c ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:34.569: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:34.569: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:34.569: INFO: netserver-1 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:34.569: INFO: alpine-nnp-false-1c058660-aa50-495f-acb1-262863728a83 started at 2023-03-16 14:54:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:34.569: INFO: Container alpine-nnp-false-1c058660-aa50-495f-acb1-262863728a83 ready: false, restart count 0 Mar 16 14:54:34.864: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:34.864: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:34.908: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 15288 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:34.908: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:34.951: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:54:35.005: INFO: pod2 started at 2023-03-16 14:54:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:35.005: INFO: pod3 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:35.005: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:35.005: INFO: execpod-affinityztffd started at 2023-03-16 14:53:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:35.005: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:54:35.005: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container coredns ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:54:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: affinity-clusterip-transition-28fjr started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:35.005: INFO: pod-server-1 started at 2023-03-16 14:54:22 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: affinity-nodeport-transition-w6wn6 started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 16 14:54:35.005: INFO: pod1 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:35.005: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: e2e-host-exec started at 2023-03-16 14:54:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container e2e-host-exec ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:54:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:35.005: INFO: test-grpc-78fcda4d-0543-4b7d-84c7-d1b98379ae7a started at 2023-03-16 14:52:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:35.005: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:35.005: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:35.005: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:35.005: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: netserver-2 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.005: INFO: host-test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.005: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:35.244: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:35.244: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:35.286: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 15287 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:35.286: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:35.333: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: test-container-pod started at 2023-03-16 14:53:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:35.386: INFO: ephemeral-containers-target-pod started at 2023-03-16 14:53:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container test-container-1 ready: true, restart count 0 Mar 16 14:54:35.386: INFO: test-container-pod started at 2023-03-16 14:54:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:54:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:54:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:35.386: INFO: kube-dns-autoscaler-7b444c59c9-wglx5 started at 2023-03-16 14:53:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:54:35.386: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:35.386: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:35.386: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:35.386: INFO: affinity-clusterip-transition-p88jm started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:35.386: INFO: affinity-nodeport-transition-j5t9c started at 2023-03-16 14:54:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container affinity-nodeport-transition ready: true, restart count 0 Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:35.386: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:54:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: exceed-active-deadline-8c5t8 started at 2023-03-16 14:54:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container c ready: true, restart count 0 Mar 16 14:54:35.386: INFO: netserver-3 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: test-container-pod started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:35.386: INFO: webserver-pod started at 2023-03-16 14:53:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:35.386: INFO: kube-proxy-mode-detector started at 2023-03-16 14:54:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:35.386: INFO: verify-service-up-exec-pod-n876n started at 2023-03-16 14:54:19 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:35.386: INFO: nodeport-update-service-d27z2 started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:54:35.386: INFO: up-down-3-gdlxs started at 2023-03-16 14:54:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:35.386: INFO: Container up-down-3 ready: true, restart count 0 Mar 16 14:54:35.653: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:54:35.653 (2.706s) < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - dump namespaces | framework.go:209 @ 03/16/23 14:54:35.653 (2.706s) > Enter [DeferCleanup (Each)] [sig-cli] Kubectl client - tear down framework | framework.go:206 @ 03/16/23 14:54:35.653 STEP: Destroying namespace "kubectl-3492" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:54:35.653 < Exit [DeferCleanup (Each)] [sig-cli] Kubectl client - tear down framework | framework.go:206 @ 03/16/23 14:54:35.697 (44ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:54:35.697 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:54:35.697 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-network\]\sServices\sshould\sserve\sendpoints\son\ssame\sport\sand\sdifferent\sprotocols$'
[FAILED] Failed to connect to Service UDP port: service is not reachable within 30s timeout on endpoint 10.0.249.58 80 over UDP protocol In [It] at: test/e2e/network/service.go:3679 @ 03/16/23 14:53:48.933from junit_01.xml
> Enter [BeforeEach] [sig-network] Services - set up framework | framework.go:191 @ 03/16/23 14:52:05.459 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:52:05.459 Mar 16 14:52:05.459: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename services - test/e2e/framework/framework.go:250 @ 03/16/23 14:52:05.46 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:52:05.585 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:52:05.666 < Exit [BeforeEach] [sig-network] Services - set up framework | framework.go:191 @ 03/16/23 14:52:05.747 (288ms) > Enter [BeforeEach] [sig-network] Services - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:52:05.747 < Exit [BeforeEach] [sig-network] Services - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:52:05.747 (0s) > Enter [BeforeEach] [sig-network] Services - test/e2e/network/service.go:764 @ 03/16/23 14:52:05.747 < Exit [BeforeEach] [sig-network] Services - test/e2e/network/service.go:764 @ 03/16/23 14:52:05.747 (0s) > Enter [It] should serve endpoints on same port and different protocols - test/e2e/network/service.go:3622 @ 03/16/23 14:52:05.747 STEP: creating service multiprotocol-test in namespace services-446 - test/e2e/network/service.go:3641 @ 03/16/23 14:52:05.747 STEP: creating pod pod1 in namespace services-446 - test/e2e/network/service.go:3667 @ 03/16/23 14:52:05.796 STEP: Creating pod pod1 in namespace services-446 - test/e2e/network/service.go:4094 @ 03/16/23 14:52:05.797 STEP: waiting up to 3m0s for service multiprotocol-test in namespace services-446 to expose endpoints map[pod1:[{tcp-port 0 80 TCP } {udp-port 0 80 UDP }]] - test/e2e/network/service.go:4286 @ 03/16/23 14:52:08.203 Mar 16 14:52:08.446: INFO: successfully validated that service multiprotocol-test in namespace services-446 exposes endpoints map[pod1:[{tcp-port 0 80 TCP } {udp-port 0 80 UDP }]] STEP: Checking if the Service forwards traffic to the TCP and UDP port - test/e2e/network/service.go:3671 @ 03/16/23 14:52:08.446 Mar 16 14:52:08.446: INFO: Creating new exec pod Mar 16 14:52:10.606: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -t -w 2 10.0.249.58 80' Mar 16 14:52:13.305: INFO: rc: 1 Mar 16 14:52:13.305: INFO: Service reachability failing with error: error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -t -w 2 10.0.249.58 80: Command stdout: stderr: + echo hostName + nc -v -t -w 2 10.0.249.58 80 nc: connect to 10.0.249.58 port 80 (tcp) timed out: Operation in progress command terminated with exit code 1 error: exit status 1 Retrying... Mar 16 14:52:15.306: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -t -w 2 10.0.249.58 80' Mar 16 14:52:15.882: INFO: stderr: "+ nc -v -t -w 2 10.0.249.58 80\nConnection to 10.0.249.58 80 port [tcp/http] succeeded!\n+ echo hostName\n" Mar 16 14:52:15.882: INFO: stdout: "HTTP/1.1 400 Bad Request\r\nContent-Type: text/plain; charset=utf-8\r\nConnection: close\r\n\r\n400 Bad Request" Mar 16 14:52:15.882: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -u -w 2 10.0.249.58 80' Mar 16 14:52:46.238: INFO: rc: 1 Mar 16 14:52:46.238: INFO: Service reachability failing with error: error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -u -w 2 10.0.249.58 80: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 Retrying... Mar 16 14:52:48.239: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -u -w 2 10.0.249.58 80' Mar 16 14:53:18.591: INFO: rc: 1 Mar 16 14:53:18.591: INFO: Service reachability failing with error: error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -u -w 2 10.0.249.58 80: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 Retrying... Mar 16 14:53:18.591: INFO: Running '/workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -u -w 2 10.0.249.58 80' Mar 16 14:53:48.933: INFO: rc: 1 Mar 16 14:53:48.933: INFO: Service reachability failing with error: error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://34.105.125.69 --kubeconfig=/workspace/.kube/config --namespace=services-446 exec execpodnsv2z -- /bin/sh -x -c echo hostName | nc -v -u -w 2 10.0.249.58 80: Command stdout: stderr: Error from server: error dialing backend: dial timeout, backstop error: exit status 1 Retrying... [FAILED] Failed to connect to Service UDP port: service is not reachable within 30s timeout on endpoint 10.0.249.58 80 over UDP protocol In [It] at: test/e2e/network/service.go:3679 @ 03/16/23 14:53:48.933 < Exit [It] should serve endpoints on same port and different protocols - test/e2e/network/service.go:3622 @ 03/16/23 14:53:48.933 (1m43.186s) > Enter [AfterEach] [sig-network] Services - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:53:48.933 Mar 16 14:53:48.933: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-network] Services - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:53:49.3 (367ms) > Enter [DeferCleanup (Each)] [sig-network] Services - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:53:49.3 < Exit [DeferCleanup (Each)] [sig-network] Services - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:53:49.3 (0s) > Enter [DeferCleanup (Each)] [sig-network] Services - dump namespaces | framework.go:209 @ 03/16/23 14:53:49.3 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:53:49.3 STEP: Collecting events from namespace "services-446". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:53:49.3 STEP: Found 8 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:53:49.444 Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:05 +0000 UTC - event for pod1: {default-scheduler } Scheduled: Successfully assigned services-446/pod1 to bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:06 +0000 UTC - event for pod1: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:06 +0000 UTC - event for pod1: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container agnhost-container Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:06 +0000 UTC - event for pod1: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container agnhost-container Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:08 +0000 UTC - event for execpodnsv2z: {default-scheduler } Scheduled: Successfully assigned services-446/execpodnsv2z to bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:09 +0000 UTC - event for execpodnsv2z: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:09 +0000 UTC - event for execpodnsv2z: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container agnhost-container Mar 16 14:53:49.444: INFO: At 2023-03-16 14:52:09 +0000 UTC - event for execpodnsv2z: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container agnhost-container Mar 16 14:53:49.543: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:53:49.543: INFO: execpodnsv2z bootstrap-e2e-minion-group-hzx6 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:08 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:10 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:10 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:08 +0000 UTC }] Mar 16 14:53:49.543: INFO: pod1 bootstrap-e2e-minion-group-hzx6 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:05 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:07 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:07 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:05 +0000 UTC }] Mar 16 14:53:49.543: INFO: Mar 16 14:53:50.015: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:53:50.119: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 12046 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:52:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:50.119: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:53:50.217: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:53:50.331: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:53:50.331: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:53:50.331: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:53:50.331: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:53:50.331: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:53:50.331: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:53:50.331: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:50.331: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:50.331: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:50.331: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:53:50.331: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.331: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:53:50.580: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:53:50.580: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:50.633: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 14124 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:53:34 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:50.633: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:50.683: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:50.806: INFO: pod-handle-http-request started at 2023-03-16 14:51:36 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:50.806: INFO: Container container-handle-8090-request ready: true, restart count 0 Mar 16 14:53:50.806: INFO: Container container-handle-9090-request ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-bd69f started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-5zscq started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-pc7f9 started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-zdw47 started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:53:50.806: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container httpd ready: true, restart count 0 Mar 16 14:53:50.806: INFO: coredns-8f5847b64-kmhw2 started at 2023-03-16 14:53:46 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-6kfgm started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-zjvd7 started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-8d75t started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:50.806: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-2p2cj started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: agnhost-host-aliases944888db-0fcf-48c9-9ecc-00f45d1723d4 started at 2023-03-16 14:53:46 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container agnhost-container ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-8mfqc started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-sgc28 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: test-rolling-update-with-lb-66756c6d5c-zbc84 started at 2023-03-16 14:53:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-jtt9j started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: netserver-0 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-6xm8w started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:50.806: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:50.806: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:50.806: INFO: sample-webhook-deployment-7497495989-cqtgd started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container sample-webhook ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-xg2pv started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-xmjvt started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-f22mc started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-l4fnt started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:50.806: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:53:50.806: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:53:50.806: INFO: execpod7b67j started at 2023-03-16 14:53:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:50.806: INFO: pod-exec-websocket-95ca6e23-c536-483f-af0e-860ba80e4abe started at 2023-03-16 14:53:33 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container main ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-ff8sk started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-k4rxw started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: netserver-0 started at 2023-03-16 14:52:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:50.806: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-sxrr2 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: ss-0 started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container webserver ready: false, restart count 0 Mar 16 14:53:50.806: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:50.806: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-499kq started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:50.806: INFO: netserver-0 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:50.806: INFO: simpletest.rc-pl22q started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:50.806: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.735: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:53:51.735: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:51.780: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:51.780: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:51.823: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:51.894: INFO: test-rolling-update-with-lb-66756c6d5c-9l8fw started at 2023-03-16 14:53:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:51.894: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-krxl7 started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:51.894: INFO: netserver-1 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:51.894: INFO: failure-1 started at 2023-03-16 14:53:42 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container failure-1 ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-9qrkd started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: nodeport-update-service-czr4d started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:53:51.894: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-t5g9z started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: httpd started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container httpd ready: false, restart count 0 Mar 16 14:53:51.894: INFO: rs-6kdwb started at 2023-03-16 14:52:35 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container donothing ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-4jgcl started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-4vr52 started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-fgwbp started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-t2m9s started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-d7w7v started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-2h9pq started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:51.894: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:51.894: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:51.894: INFO: execpodf4mm6 started at 2023-03-16 14:51:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-fbldn started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-lr6gv started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-xp4lk started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-j2l76 started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-xdlsg started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-x6zg4 started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-27mjq started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-hw7gt started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: affinity-clusterip-transition-7prln started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:53:51.894: INFO: pause-pod-0 started at 2023-03-16 14:51:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-gdfnq started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-dnsjr started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-tkfr7 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-b4fjn started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-vqbqf started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:51.894: INFO: netserver-1 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-kx22m started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:51.894: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:51.894: INFO: execpodnsv2z started at 2023-03-16 14:52:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:51.894: INFO: pod1 started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:51.894: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-j9j4h started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:51.894: INFO: pod1 started at 2023-03-16 14:52:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:51.894: INFO: simpletest.rc-gzqqs started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:51.894: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.100: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:53:55.100: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:53:55.146: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 6652 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:55.146: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:53:55.198: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:53:55.254: INFO: nodeport-test-dg97z started at 2023-03-16 14:53:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nodeport-test ready: true, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-bspw7 started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-tpzh8 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-s8g59 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-49qjj started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-mxcwf started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:55.254: INFO: pod2 started at 2023-03-16 14:52:33 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:55.254: INFO: execpod-affinityztffd started at 2023-03-16 14:53:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:55.254: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:53:55.254: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-mmsd7 started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: affinity-clusterip-transition-28fjr started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:53:55.254: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:55.254: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:55.254: INFO: image-pull-testfe18163c-12f1-4b96-8b6e-ddaa0d843e9a started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container image-pull-test ready: false, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-dzgbq started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: pod1 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container agnhost ready: false, restart count 0 Mar 16 14:53:55.254: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:55.254: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:52:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:55.254: INFO: netserver-2 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:55.254: INFO: test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:55.254: INFO: e2e-net-exec started at 2023-03-16 14:53:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 16 14:53:55.254: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:55.254: INFO: verify-service-up-exec-pod-r45vt started at 2023-03-16 14:52:22 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:55.254: INFO: test-grpc-78fcda4d-0543-4b7d-84c7-d1b98379ae7a started at 2023-03-16 14:52:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:55.254: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:55.254: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:55.254: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:55.254: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:55.254: INFO: netserver-2 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:55.254: INFO: host-test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-zrkx4 started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-k4lp7 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:55.254: INFO: simpletest.rc-2g7pw started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:55.254: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.306: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:53:56.306: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:53:56.349: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 14084 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:53:33 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:33 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:33 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:33 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:53:33 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:53:56.349: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:53:56.427: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:53:56.497: INFO: kube-dns-autoscaler-7b444c59c9-wglx5 started at 2023-03-16 14:53:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:53:56.498: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:53:56.498: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:53:56.498: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-k8gdv started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-54rxj started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: affinity-clusterip-transition-p88jm started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:53:56.498: INFO: nodeport-test-zzq4p started at 2023-03-16 14:53:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nodeport-test ready: true, restart count 0 Mar 16 14:53:56.498: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-jl262 started at 2023-03-16 14:53:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-nv797 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:56.498: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:53:56.498: INFO: execpodz4xt8 started at 2023-03-16 14:53:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-slpv6 started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: netserver-3 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:56.498: INFO: test-container-pod started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:56.498: INFO: nodeport-update-service-d27z2 started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-x895z started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: webserver-pod started at 2023-03-16 14:53:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-dps2x started at 2023-03-16 14:53:50 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-qbqf4 started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-g48p6 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-7pd2n started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-qcmmk started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: test-container-pod started at 2023-03-16 14:53:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:56.498: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:56.498: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-hkw9j started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: netserver-3 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:56.498: INFO: test-grpc-31967f11-8259-4e30-a7c5-d8b1d0d6ad49 started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:56.498: INFO: ephemeral-containers-target-pod started at 2023-03-16 14:53:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container test-container-1 ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-8fpqn started at 2023-03-16 14:53:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-xqjpg started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container webserver ready: true, restart count 0 Mar 16 14:53:56.498: INFO: coredns-8f5847b64-22spj started at 2023-03-16 14:53:46 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container coredns ready: true, restart count 0 Mar 16 14:53:56.498: INFO: simpletest.rc-4p2br started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container nginx ready: false, restart count 0 Mar 16 14:53:56.498: INFO: test-rolling-update-with-lb-66756c6d5c-jfk8b started at 2023-03-16 14:53:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:53:56.498: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:53:57.975: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:53:57.975 (8.675s) < Exit [DeferCleanup (Each)] [sig-network] Services - dump namespaces | framework.go:209 @ 03/16/23 14:53:57.975 (8.675s) > Enter [DeferCleanup (Each)] [sig-network] Services - tear down framework | framework.go:206 @ 03/16/23 14:53:57.975 STEP: Destroying namespace "services-446" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:53:57.975 < Exit [DeferCleanup (Each)] [sig-network] Services - tear down framework | framework.go:206 @ 03/16/23 14:53:58.402 (427ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:53:58.402 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:53:58.402 (0s)
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\sEphemeral\sContainers\s\[NodeConformance\]\swill\sstart\san\sephemeral\scontainer\sin\san\sexisting\spod\s\[Conformance\]$'
[FAILED] failed to execute command in pod ephemeral-containers-target-pod, container debugger: error dialing backend: dial timeout, backstop: error dialing backend: dial timeout, backstop In [It] at: test/e2e/framework/pod/exec_util.go:107 @ 03/16/23 14:54:02.209from junit_01.xml
> Enter [BeforeEach] [sig-node] Ephemeral Containers [NodeConformance] - set up framework | framework.go:191 @ 03/16/23 14:53:27.221 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:53:27.221 Mar 16 14:53:27.221: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename ephemeral-containers-test - test/e2e/framework/framework.go:250 @ 03/16/23 14:53:27.222 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:53:27.409 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:53:27.492 < Exit [BeforeEach] [sig-node] Ephemeral Containers [NodeConformance] - set up framework | framework.go:191 @ 03/16/23 14:53:27.635 (414ms) > Enter [BeforeEach] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:53:27.635 < Exit [BeforeEach] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:53:27.635 (0s) > Enter [BeforeEach] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/common/node/ephemeral_containers.go:39 @ 03/16/23 14:53:27.635 < Exit [BeforeEach] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/common/node/ephemeral_containers.go:39 @ 03/16/23 14:53:27.635 (0s) > Enter [It] will start an ephemeral container in an existing pod [Conformance] - test/e2e/common/node/ephemeral_containers.go:46 @ 03/16/23 14:53:27.635 STEP: creating a target pod - test/e2e/common/node/ephemeral_containers.go:47 @ 03/16/23 14:53:27.635 STEP: adding an ephemeral container - test/e2e/common/node/ephemeral_containers.go:62 @ 03/16/23 14:53:29.902 STEP: checking pod container endpoints - test/e2e/common/node/ephemeral_containers.go:76 @ 03/16/23 14:53:32.079 Mar 16 14:53:32.079: INFO: ExecWithOptions {Command:[/bin/echo marco] Namespace:ephemeral-containers-test-3299 PodName:ephemeral-containers-target-pod ContainerName:debugger Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 16 14:53:32.079: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:53:32.080: INFO: ExecWithOptions: Clientset creation Mar 16 14:53:32.080: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/ephemeral-containers-test-3299/pods/ephemeral-containers-target-pod/exec?command=%2Fbin%2Fecho&command=marco&container=debugger&container=debugger&stderr=true&stdout=true) Mar 16 14:54:02.209: INFO: Exec stderr: "" Mar 16 14:54:02.209: INFO: Unexpected error: failed to execute command in pod ephemeral-containers-target-pod, container debugger: error dialing backend: dial timeout, backstop: <*errors.StatusError | 0xc003045680>: error dialing backend: dial timeout, backstop { ErrStatus: code: 500 message: 'error dialing backend: dial timeout, backstop' metadata: {} status: Failure, } [FAILED] failed to execute command in pod ephemeral-containers-target-pod, container debugger: error dialing backend: dial timeout, backstop: error dialing backend: dial timeout, backstop In [It] at: test/e2e/framework/pod/exec_util.go:107 @ 03/16/23 14:54:02.209 < Exit [It] will start an ephemeral container in an existing pod [Conformance] - test/e2e/common/node/ephemeral_containers.go:46 @ 03/16/23 14:54:02.209 (34.574s) > Enter [AfterEach] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:54:02.209 Mar 16 14:54:02.209: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:54:02.258 (48ms) > Enter [DeferCleanup (Each)] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:54:02.258 < Exit [DeferCleanup (Each)] [sig-node] Ephemeral Containers [NodeConformance] - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:54:02.258 (0s) > Enter [DeferCleanup (Each)] [sig-node] Ephemeral Containers [NodeConformance] - dump namespaces | framework.go:209 @ 03/16/23 14:54:02.258 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:54:02.258 STEP: Collecting events from namespace "ephemeral-containers-test-3299". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:54:02.258 STEP: Found 7 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:54:02.3 Mar 16 14:54:02.300: INFO: At 2023-03-16 14:53:27 +0000 UTC - event for ephemeral-containers-target-pod: {default-scheduler } Scheduled: Successfully assigned ephemeral-containers-test-3299/ephemeral-containers-target-pod to bootstrap-e2e-minion-group-z4ns Mar 16 14:54:02.300: INFO: At 2023-03-16 14:53:28 +0000 UTC - event for ephemeral-containers-target-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 16 14:54:02.300: INFO: At 2023-03-16 14:53:28 +0000 UTC - event for ephemeral-containers-target-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container test-container-1 Mar 16 14:54:02.300: INFO: At 2023-03-16 14:53:28 +0000 UTC - event for ephemeral-containers-target-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container test-container-1 Mar 16 14:54:02.300: INFO: At 2023-03-16 14:53:30 +0000 UTC - event for ephemeral-containers-target-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 16 14:54:02.300: INFO: At 2023-03-16 14:53:30 +0000 UTC - event for ephemeral-containers-target-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container debugger Mar 16 14:54:02.300: INFO: At 2023-03-16 14:53:30 +0000 UTC - event for ephemeral-containers-target-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container debugger Mar 16 14:54:02.342: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:54:02.342: INFO: ephemeral-containers-target-pod bootstrap-e2e-minion-group-z4ns Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:53:27 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:53:29 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:53:29 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:53:27 +0000 UTC }] Mar 16 14:54:02.342: INFO: Mar 16 14:54:02.456: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:54:02.511: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 12046 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:52:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:02.512: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:54:02.558: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:54:02.615: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:02.615: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:02.615: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:02.615: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:54:02.615: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:54:02.615: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:54:02.615: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:54:02.615: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:54:02.615: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:54:02.615: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:54:02.615: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:02.615: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:54:02.809: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:54:02.809: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:02.854: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 14124 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:53:34 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:53:34 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:02.855: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:02.904: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:03.015: INFO: ss-0 started at 2023-03-16 14:53:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.015: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:03.015: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:03.016: INFO: e2e-test-httpd-pod started at 2023-03-16 14:53:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container e2e-test-httpd-pod ready: false, restart count 0 Mar 16 14:54:03.016: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:03.016: INFO: netserver-0 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:03.016: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:03.016: INFO: pod-handle-http-request started at 2023-03-16 14:51:36 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:03.016: INFO: Container container-handle-8090-request ready: true, restart count 0 Mar 16 14:54:03.016: INFO: Container container-handle-9090-request ready: true, restart count 0 Mar 16 14:54:03.016: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:54:03.016: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container httpd ready: true, restart count 0 Mar 16 14:54:03.016: INFO: coredns-8f5847b64-kmhw2 started at 2023-03-16 14:53:46 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container coredns ready: true, restart count 0 Mar 16 14:54:03.016: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:03.016: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:54:03.016: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:03.016: INFO: netserver-0 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:03.016: INFO: test-rolling-update-with-lb-66756c6d5c-zbc84 started at 2023-03-16 14:53:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:03.016: INFO: var-expansion-66c868a6-f00f-4365-8b38-cfd78c7d9cea started at 2023-03-16 14:54:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container dapi-container ready: false, restart count 0 Mar 16 14:54:03.016: INFO: my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac-2mkz6 started at 2023-03-16 14:53:51 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container my-hostname-private-41ae7519-df22-4ed9-be32-cb435c1585ac ready: false, restart count 0 Mar 16 14:54:03.016: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:03.016: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:03.016: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:03.016: INFO: sample-webhook-deployment-7497495989-cqtgd started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container sample-webhook ready: true, restart count 0 Mar 16 14:54:03.016: INFO: pod-exec-websocket-95ca6e23-c536-483f-af0e-860ba80e4abe started at 2023-03-16 14:53:33 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container main ready: true, restart count 0 Mar 16 14:54:03.016: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:03.016: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:54:03.016: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:54:03.016: INFO: execpod7b67j started at 2023-03-16 14:53:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:03.016: INFO: netserver-0 started at 2023-03-16 14:52:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:03.016: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:05.571: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:54:05.571: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:05.613: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:05.614: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:05.660: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:05.737: INFO: pod1 started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:05.737: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:05.737: INFO: busybox-privileged-true-d46d04ab-8878-4f9f-8515-104941036d87 started at 2023-03-16 14:53:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container busybox-privileged-true-d46d04ab-8878-4f9f-8515-104941036d87 ready: false, restart count 0 Mar 16 14:54:05.737: INFO: test-rolling-update-with-lb-66756c6d5c-9l8fw started at 2023-03-16 14:53:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:05.737: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:05.737: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:05.737: INFO: netserver-1 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:05.737: INFO: nodeport-update-service-czr4d started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:54:05.737: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:05.737: INFO: httpd started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container httpd ready: true, restart count 0 Mar 16 14:54:05.737: INFO: rs-6kdwb started at 2023-03-16 14:52:35 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container donothing ready: false, restart count 0 Mar 16 14:54:05.737: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:05.737: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:05.737: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:05.737: INFO: execpodf4mm6 started at 2023-03-16 14:51:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:05.737: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:05.737: INFO: implicit-root-uid started at 2023-03-16 14:54:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container implicit-root-uid ready: false, restart count 0 Mar 16 14:54:05.737: INFO: affinity-clusterip-transition-7prln started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:05.737: INFO: pause-pod-0 started at 2023-03-16 14:51:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:05.737: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container coredns ready: true, restart count 0 Mar 16 14:54:05.737: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:05.737: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:05.737: INFO: netserver-1 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:05.737: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.230: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:54:07.230: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:07.277: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 15288 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:13ac9b3c476d7290a4451d65d09d6016a2cb89836ffbfa4eae55b72731a22080 registry.k8s.io/build-image/distroless-iptables:v0.2.2],SizeBytes:7729580,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:07.277: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:07.324: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:54:07.392: INFO: pod2 started at 2023-03-16 14:54:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:07.392: INFO: pod3 started at 2023-03-16 14:54:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost ready: false, restart count 0 Mar 16 14:54:07.392: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:07.392: INFO: pod2 started at 2023-03-16 14:52:33 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:07.392: INFO: execpod-affinityztffd started at 2023-03-16 14:53:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:07.392: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.392: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:54:07.392: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container coredns ready: true, restart count 0 Mar 16 14:54:07.392: INFO: affinity-clusterip-transition-28fjr started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:07.392: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.392: INFO: pod1 started at 2023-03-16 14:53:48 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:07.392: INFO: test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.392: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:07.392: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:52:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:07.392: INFO: netserver-2 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.392: INFO: e2e-net-exec started at 2023-03-16 14:53:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container e2e-net-exec ready: true, restart count 0 Mar 16 14:54:07.392: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.392: INFO: verify-service-up-exec-pod-r45vt started at 2023-03-16 14:52:22 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:07.392: INFO: test-grpc-78fcda4d-0543-4b7d-84c7-d1b98379ae7a started at 2023-03-16 14:52:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:07.392: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:07.392: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:07.392: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:07.392: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.392: INFO: netserver-2 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:07.392: INFO: host-test-container-pod started at 2023-03-16 14:52:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:07.392: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:10.173: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:54:10.173: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:10.220: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 15287 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:54:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:54:03 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:54:10.220: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:10.266: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:54:10.343: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:10.343: INFO: test-container-pod started at 2023-03-16 14:53:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:10.343: INFO: netserver-3 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:10.343: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:54:10.343: INFO: ephemeral-containers-target-pod started at 2023-03-16 14:53:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container test-container-1 ready: true, restart count 0 Mar 16 14:54:10.343: INFO: sample-webhook-deployment-7497495989-9hpj4 started at 2023-03-16 14:53:59 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container sample-webhook ready: true, restart count 0 Mar 16 14:54:10.343: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:10.343: INFO: test-rolling-update-with-lb-66756c6d5c-jfk8b started at 2023-03-16 14:53:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:54:10.343: INFO: kube-dns-autoscaler-7b444c59c9-wglx5 started at 2023-03-16 14:53:43 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:54:10.343: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:54:10.343: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:54:10.343: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:54:10.343: INFO: ss-1 started at 2023-03-16 14:54:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:10.343: INFO: affinity-clusterip-transition-p88jm started at 2023-03-16 14:53:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container affinity-clusterip-transition ready: true, restart count 0 Mar 16 14:54:10.343: INFO: failed-jobs-history-limit-27982974-ch62c started at 2023-03-16 14:54:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container c ready: false, restart count 1 Mar 16 14:54:10.343: INFO: terminate-cmd-rpa56a6af25-9e57-42a6-83e1-3e542c4443bc started at 2023-03-16 14:54:09 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container terminate-cmd-rpa ready: false, restart count 0 Mar 16 14:54:10.343: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:10.343: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:54:10.343: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:54:10.343: INFO: netserver-3 started at 2023-03-16 14:54:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: false, restart count 0 Mar 16 14:54:10.343: INFO: netserver-3 started at 2023-03-16 14:53:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:10.343: INFO: test-container-pod started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container webserver ready: true, restart count 0 Mar 16 14:54:10.343: INFO: webserver-pod started at 2023-03-16 14:53:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:54:10.343: INFO: nodeport-update-service-d27z2 started at 2023-03-16 14:52:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:54:10.343: INFO: Container nodeport-update-service ready: true, restart count 0 Mar 16 14:54:13.607: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:54:13.607 (11.349s) < Exit [DeferCleanup (Each)] [sig-node] Ephemeral Containers [NodeConformance] - dump namespaces | framework.go:209 @ 03/16/23 14:54:13.607 (11.349s) > Enter [DeferCleanup (Each)] [sig-node] Ephemeral Containers [NodeConformance] - tear down framework | framework.go:206 @ 03/16/23 14:54:13.607 STEP: Destroying namespace "ephemeral-containers-test-3299" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:54:13.607 < Exit [DeferCleanup (Each)] [sig-node] Ephemeral Containers [NodeConformance] - tear down framework | framework.go:206 @ 03/16/23 14:54:13.652 (45ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:54:13.652 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:54:13.652 (0s)
Find ephemeral-containers-target-pod, mentions in log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\sKubeletManagedEtcHosts\sshould\stest\skubelet\smanaged\s\/etc\/hosts\sfile\s\[LinuxOnly\]\s\[NodeConformance\]\s\[Conformance\]$'
[FAILED] failed to execute command in pod test-pod, container busybox-2: error dialing backend: dial timeout, backstop: error dialing backend: dial timeout, backstop In [It] at: test/e2e/framework/pod/exec_util.go:107 @ 03/16/23 14:52:40.633from junit_01.xml
> Enter [BeforeEach] [sig-node] KubeletManagedEtcHosts - set up framework | framework.go:191 @ 03/16/23 14:52:04.657 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:52:04.657 Mar 16 14:52:04.657: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename e2e-kubelet-etc-hosts - test/e2e/framework/framework.go:250 @ 03/16/23 14:52:04.659 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:52:04.782 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:52:04.862 < Exit [BeforeEach] [sig-node] KubeletManagedEtcHosts - set up framework | framework.go:191 @ 03/16/23 14:52:04.947 (290ms) > Enter [BeforeEach] [sig-node] KubeletManagedEtcHosts - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:52:04.947 < Exit [BeforeEach] [sig-node] KubeletManagedEtcHosts - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:52:04.947 (0s) > Enter [It] should test kubelet managed /etc/hosts file [LinuxOnly] [NodeConformance] [Conformance] - test/e2e/common/node/kubelet_etc_hosts.go:64 @ 03/16/23 14:52:04.947 STEP: Setting up the test - test/e2e/common/node/kubelet_etc_hosts.go:65 @ 03/16/23 14:52:04.947 STEP: Creating hostNetwork=false pod - test/e2e/common/node/kubelet_etc_hosts.go:87 @ 03/16/23 14:52:04.947 STEP: Creating hostNetwork=true pod - test/e2e/common/node/kubelet_etc_hosts.go:90 @ 03/16/23 14:52:07.17 STEP: Running the test - test/e2e/common/node/kubelet_etc_hosts.go:68 @ 03/16/23 14:52:09.486 STEP: Verifying /etc/hosts of container is kubelet-managed for pod with hostNetwork=false - test/e2e/common/node/kubelet_etc_hosts.go:74 @ 03/16/23 14:52:09.486 Mar 16 14:52:09.486: INFO: ExecWithOptions {Command:[cat /etc/hosts] Namespace:e2e-kubelet-etc-hosts-7419 PodName:test-pod ContainerName:busybox-1 Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 16 14:52:09.486: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:52:09.487: INFO: ExecWithOptions: Clientset creation Mar 16 14:52:09.487: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/e2e-kubelet-etc-hosts-7419/pods/test-pod/exec?command=cat&command=%2Fetc%2Fhosts&container=busybox-1&container=busybox-1&stderr=true&stdout=true) Mar 16 14:52:09.809: INFO: Exec stderr: "" Mar 16 14:52:09.809: INFO: ExecWithOptions {Command:[cat /etc/hosts-original] Namespace:e2e-kubelet-etc-hosts-7419 PodName:test-pod ContainerName:busybox-1 Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 16 14:52:09.809: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:52:09.810: INFO: ExecWithOptions: Clientset creation Mar 16 14:52:09.810: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/e2e-kubelet-etc-hosts-7419/pods/test-pod/exec?command=cat&command=%2Fetc%2Fhosts-original&container=busybox-1&container=busybox-1&stderr=true&stdout=true) Mar 16 14:52:10.152: INFO: Exec stderr: "" Mar 16 14:52:10.152: INFO: ExecWithOptions {Command:[cat /etc/hosts] Namespace:e2e-kubelet-etc-hosts-7419 PodName:test-pod ContainerName:busybox-2 Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 16 14:52:10.152: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:52:10.153: INFO: ExecWithOptions: Clientset creation Mar 16 14:52:10.153: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/e2e-kubelet-etc-hosts-7419/pods/test-pod/exec?command=cat&command=%2Fetc%2Fhosts&container=busybox-2&container=busybox-2&stderr=true&stdout=true) Mar 16 14:52:10.498: INFO: Exec stderr: "" Mar 16 14:52:10.498: INFO: ExecWithOptions {Command:[cat /etc/hosts-original] Namespace:e2e-kubelet-etc-hosts-7419 PodName:test-pod ContainerName:busybox-2 Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 16 14:52:10.498: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:52:10.499: INFO: ExecWithOptions: Clientset creation Mar 16 14:52:10.499: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/e2e-kubelet-etc-hosts-7419/pods/test-pod/exec?command=cat&command=%2Fetc%2Fhosts-original&container=busybox-2&container=busybox-2&stderr=true&stdout=true) Mar 16 14:52:40.632: INFO: Exec stderr: "" Mar 16 14:52:40.633: INFO: Unexpected error: failed to execute command in pod test-pod, container busybox-2: error dialing backend: dial timeout, backstop: <*errors.StatusError | 0xc00325a500>: error dialing backend: dial timeout, backstop { ErrStatus: code: 500 message: 'error dialing backend: dial timeout, backstop' metadata: {} status: Failure, } [FAILED] failed to execute command in pod test-pod, container busybox-2: error dialing backend: dial timeout, backstop: error dialing backend: dial timeout, backstop In [It] at: test/e2e/framework/pod/exec_util.go:107 @ 03/16/23 14:52:40.633 < Exit [It] should test kubelet managed /etc/hosts file [LinuxOnly] [NodeConformance] [Conformance] - test/e2e/common/node/kubelet_etc_hosts.go:64 @ 03/16/23 14:52:40.633 (35.686s) > Enter [AfterEach] [sig-node] KubeletManagedEtcHosts - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:52:40.633 Mar 16 14:52:40.633: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-node] KubeletManagedEtcHosts - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:52:40.679 (46ms) > Enter [DeferCleanup (Each)] [sig-node] KubeletManagedEtcHosts - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:52:40.679 < Exit [DeferCleanup (Each)] [sig-node] KubeletManagedEtcHosts - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:52:40.679 (0s) > Enter [DeferCleanup (Each)] [sig-node] KubeletManagedEtcHosts - dump namespaces | framework.go:209 @ 03/16/23 14:52:40.679 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:52:40.679 STEP: Collecting events from namespace "e2e-kubelet-etc-hosts-7419". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:52:40.679 STEP: Found 17 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:52:40.756 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:04 +0000 UTC - event for test-pod: {default-scheduler } Scheduled: Successfully assigned e2e-kubelet-etc-hosts-7419/test-pod to bootstrap-e2e-minion-group-z4ns Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:05 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container busybox-2 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:05 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:05 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:05 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container busybox-1 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:05 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container busybox-1 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:06 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container busybox-2 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:06 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:06 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Created: Created container busybox-3 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:06 +0000 UTC - event for test-pod: {kubelet bootstrap-e2e-minion-group-z4ns} Started: Started container busybox-3 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:07 +0000 UTC - event for test-host-network-pod: {default-scheduler } Scheduled: Successfully assigned e2e-kubelet-etc-hosts-7419/test-host-network-pod to bootstrap-e2e-minion-group-2p9b Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:08 +0000 UTC - event for test-host-network-pod: {kubelet bootstrap-e2e-minion-group-2p9b} Started: Started container busybox-2 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:08 +0000 UTC - event for test-host-network-pod: {kubelet bootstrap-e2e-minion-group-2p9b} Created: Created container busybox-1 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:08 +0000 UTC - event for test-host-network-pod: {kubelet bootstrap-e2e-minion-group-2p9b} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:08 +0000 UTC - event for test-host-network-pod: {kubelet bootstrap-e2e-minion-group-2p9b} Created: Created container busybox-2 Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:08 +0000 UTC - event for test-host-network-pod: {kubelet bootstrap-e2e-minion-group-2p9b} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:52:40.756: INFO: At 2023-03-16 14:52:08 +0000 UTC - event for test-host-network-pod: {kubelet bootstrap-e2e-minion-group-2p9b} Started: Started container busybox-1 Mar 16 14:52:40.803: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:52:40.803: INFO: test-host-network-pod bootstrap-e2e-minion-group-2p9b Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:07 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:09 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:09 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:07 +0000 UTC }] Mar 16 14:52:40.803: INFO: test-pod bootstrap-e2e-minion-group-z4ns Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:04 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:06 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:06 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:52:04 +0000 UTC }] Mar 16 14:52:40.803: INFO: Mar 16 14:52:41.144: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:52:41.189: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 660 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:47:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:47:47 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:41.189: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:52:41.234: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:52:41.301: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:52:41.301: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:52:41.301: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:52:41.301: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:52:41.301: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:52:41.301: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:52:41.301: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:52:41.301: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.301: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:52:41.301: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:41.301: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:41.301: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:41.509: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:52:41.509: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:41.553: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 8265 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2p9b kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:49:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-2p9b,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:49:58 +0000 UTC,LastTransitionTime:2023-03-16 14:49:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.147.225,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2p9b.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e925bb2c4733de08eb82cd2e9eced3fd,SystemUUID:e925bb2c-4733-de08-eb82-cd2e9eced3fd,BootID:4c24cad7-0e00-47c4-868b-3bca81bf0834,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:41.553: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:41.600: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:41.666: INFO: konnectivity-agent-dqsxb started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:41.666: INFO: pod-handle-http-request started at 2023-03-16 14:51:36 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:41.666: INFO: Container container-handle-8090-request ready: true, restart count 0 Mar 16 14:52:41.666: INFO: Container container-handle-9090-request ready: true, restart count 0 Mar 16 14:52:41.666: INFO: test-host-network-pod started at 2023-03-16 14:52:07 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:41.666: INFO: Container busybox-1 ready: true, restart count 0 Mar 16 14:52:41.666: INFO: Container busybox-2 ready: true, restart count 0 Mar 16 14:52:41.666: INFO: netserver-0 started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:41.666: INFO: volume-snapshot-controller-0 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container volume-snapshot-controller ready: true, restart count 0 Mar 16 14:52:41.666: INFO: httpd started at 2023-03-16 14:48:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container httpd ready: true, restart count 0 Mar 16 14:52:41.666: INFO: run-test-3 started at 2023-03-16 14:52:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container run-test-3 ready: true, restart count 0 Mar 16 14:52:41.666: INFO: busybox-a5a35a8e-81ce-4aed-8ded-436003287c63 started at 2023-03-16 14:49:44 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container busybox ready: true, restart count 0 Mar 16 14:52:41.666: INFO: netserver-0 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:41.666: INFO: proxy-service-n5qdm-j8crl started at 2023-03-16 14:49:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container proxy-service-n5qdm ready: true, restart count 0 Mar 16 14:52:41.666: INFO: test-rolling-update-with-lb-69fcddc54f-6qbjs started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:41.666: INFO: kube-proxy-bootstrap-e2e-minion-group-2p9b started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:41.666: INFO: rs-q4dgq started at 2023-03-16 14:52:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container donothing ready: false, restart count 0 Mar 16 14:52:41.666: INFO: netserver-0 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:41.666: INFO: metadata-proxy-v0.1-n5n9q started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:41.666: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:41.666: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:41.666: INFO: pod-client started at 2023-03-16 14:52:16 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container pod-client ready: true, restart count 0 Mar 16 14:52:41.666: INFO: metrics-server-v0.5.2-57d47cbf5-rx575 started at 2023-03-16 14:47:39 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:41.666: INFO: Container metrics-server ready: true, restart count 0 Mar 16 14:52:41.666: INFO: Container metrics-server-nanny ready: true, restart count 0 Mar 16 14:52:41.666: INFO: netserver-0 started at 2023-03-16 14:52:20 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:41.666: INFO: up-down-2-qw7fd started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:41.666: INFO: netserver-0 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:41.666: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:41.887: INFO: Latency metrics for node bootstrap-e2e-minion-group-2p9b Mar 16 14:52:41.887: INFO: Logging node info for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:41.932: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-hzx6 65a45cf3-e702-43bb-96d3-ea8c678dec27 5346 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-hzx6 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:02 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:50:29 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-hzx6,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:50:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:02 +0000 UTC,LastTransitionTime:2023-03-16 14:47:28 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:50:29 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.4,},NodeAddress{Type:ExternalIP,Address:34.168.174.6,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-hzx6.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:415976cf2ffe0074667e25f79ef568e2,SystemUUID:415976cf-2ffe-0074-667e-25f79ef568e2,BootID:c36f3fa1-8821-4652-b8a4-02ccf161a124,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:41.932: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:41.981: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:42.047: INFO: pause-pod-0 started at 2023-03-16 14:51:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.047: INFO: probe-test-b81d4fc4-f22a-4b94-80a5-af7def1bb6a9 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container probe-test-b81d4fc4-f22a-4b94-80a5-af7def1bb6a9 ready: false, restart count 0 Mar 16 14:52:42.047: INFO: pod2 started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container container1 ready: true, restart count 0 Mar 16 14:52:42.047: INFO: coredns-8f5847b64-vtg59 started at 2023-03-16 14:47:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container coredns ready: true, restart count 0 Mar 16 14:52:42.047: INFO: netserver-1 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.047: INFO: execpodnsv2z started at 2023-03-16 14:52:08 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.047: INFO: test-container-pod started at 2023-03-16 14:52:13 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.047: INFO: netserver-1 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:42.047: INFO: pod1 started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.047: INFO: up-down-2-lkn77 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:42.047: INFO: pod1 started at 2023-03-16 14:52:12 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container container1 ready: true, restart count 0 Mar 16 14:52:42.047: INFO: pod1 started at 2023-03-16 14:52:05 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.047: INFO: test-rolling-update-with-lb-69fcddc54f-chjs7 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:42.047: INFO: konnectivity-agent-7jml7 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:42.047: INFO: kube-proxy-bootstrap-e2e-minion-group-hzx6 started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:42.047: INFO: netserver-1 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.047: INFO: externalname-service-6q82d started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:52:42.047: INFO: rs-6kdwb started at 2023-03-16 14:52:35 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container donothing ready: false, restart count 0 Mar 16 14:52:42.047: INFO: metadata-proxy-v0.1-7b8wh started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:42.047: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:42.047: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:42.047: INFO: execpodf4mm6 started at 2023-03-16 14:51:57 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.047: INFO: successful-jobs-history-limit-27982972-5xjm6 started at 2023-03-16 14:52:00 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container c ready: false, restart count 0 Mar 16 14:52:42.047: INFO: netserver-1 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.047: INFO: pod-server-1 started at 2023-03-16 14:52:18 +0000 UTC (1+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Init container init ready: true, restart count 0 Mar 16 14:52:42.047: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.047: INFO: netserver-1 started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:42.047: INFO: pod-client started at 2023-03-16 14:52:27 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.047: INFO: Container pod-client ready: true, restart count 0 Mar 16 14:52:42.251: INFO: Latency metrics for node bootstrap-e2e-minion-group-hzx6 Mar 16 14:52:42.251: INFO: Logging node info for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:42.295: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-kslm f70910b1-93dc-4c81-8bcd-c7d18cdbd1c7 6652 0 2023-03-16 14:47:24 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-kslm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2023-03-16 14:47:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.4.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2023-03-16 14:50:06 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:51:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.4.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-kslm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.4.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:50:05 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:06 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:51:00 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.6,},NodeAddress{Type:ExternalIP,Address:34.168.130.18,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-kslm.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:63442ba26c81d3d3ef4987c00233e494,SystemUUID:63442ba2-6c81-d3d3-ef49-87c00233e494,BootID:0bd16c3d-c68f-479d-837d-c4ede6247b61,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:a0ead06651cf580044aeb0a0feba63591858fb2e43ade8c9dea45a6a89ae7e5e registry.k8s.io/coredns/coredns:v1.10.1],SizeBytes:16190758,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:42.295: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:42.341: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-kslm Mar 16 14:52:42.404: INFO: metadata-proxy-v0.1-b4qnh started at 2023-03-16 14:47:25 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:42.404: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:42.404: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:42.404: INFO: test-container-pod started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.404: INFO: execpodpplld started at 2023-03-16 14:51:55 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.404: INFO: externalname-service-22tpf started at 2023-03-16 14:51:52 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container externalname-service ready: true, restart count 0 Mar 16 14:52:42.404: INFO: kube-dns-autoscaler-7b444c59c9-2q9j6 started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container autoscaler ready: true, restart count 0 Mar 16 14:52:42.404: INFO: konnectivity-agent-dvz4b started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:42.404: INFO: pod2 started at 2023-03-16 14:52:33 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.404: INFO: busybox-privileged-false-f9927aff-11e1-498b-85f6-1e0c0884dd8d started at 2023-03-16 14:52:34 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container busybox-privileged-false-f9927aff-11e1-498b-85f6-1e0c0884dd8d ready: false, restart count 0 Mar 16 14:52:42.404: INFO: netserver-2 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.404: INFO: l7-default-backend-856d874f49-sbbmg started at 2023-03-16 14:47:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container default-http-backend ready: true, restart count 0 Mar 16 14:52:42.404: INFO: coredns-8f5847b64-kpk5h started at 2023-03-16 14:47:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container coredns ready: true, restart count 0 Mar 16 14:52:42.404: INFO: externalip-test-wljnd started at 2023-03-16 14:52:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container externalip-test ready: true, restart count 0 Mar 16 14:52:42.404: INFO: test-recreate-deployment-54757ffd6c-57c2k started at 2023-03-16 14:52:41 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container httpd ready: false, restart count 0 Mar 16 14:52:42.404: INFO: netserver-2 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.404: INFO: netserver-2 started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:42.404: INFO: kube-proxy-bootstrap-e2e-minion-group-kslm started at 2023-03-16 14:47:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:42.404: INFO: verify-service-up-host-exec-pod started at 2023-03-16 14:52:17 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.404: INFO: netserver-2 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.404: INFO: execpod-zzscz started at 2023-03-16 14:52:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.404: INFO: netserver-2 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.404: INFO: verify-service-up-exec-pod-r45vt started at 2023-03-16 14:52:22 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.404: INFO: liveness-7327daba-341e-4055-9fdc-ee3d9f5b31f6 started at 2023-03-16 14:52:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.404: INFO: test-grpc-78fcda4d-0543-4b7d-84c7-d1b98379ae7a started at 2023-03-16 14:52:15 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.404: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:42.642: INFO: Latency metrics for node bootstrap-e2e-minion-group-kslm Mar 16 14:52:42.642: INFO: Logging node info for node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:42.684: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-z4ns 2782be88-0c75-44f1-9752-2aab14deefed 11234 0 2023-03-16 14:47:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-z4ns kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2023-03-16 14:50:10 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2023-03-16 14:52:31 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-minion-group-z4ns,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:50:08 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2023-03-16 14:50:10 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:25 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:31 +0000 UTC,LastTransitionTime:2023-03-16 14:47:25 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.3,},NodeAddress{Type:ExternalIP,Address:34.127.53.138,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-z4ns.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7770bf856c193183b36828e68cfba790,SystemUUID:7770bf85-6c19-3183-b368-28e68cfba790,BootID:5e89b506-4e8a-4d17-b0cb-4a1d5ad15c59,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:72583008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/sample-apiserver@sha256:8d70890151aa5d096f331cb9da1b9cd5be0412b7363fe67b5c3befdcaa2a28d0 registry.k8s.io/e2e-test-images/sample-apiserver:1.17.7],SizeBytes:25667066,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:939c42e815e6b6af3181f074652c0d18fe429fcee9b49c1392aee7e92887cfef registry.k8s.io/kas-network-proxy/proxy-agent:v0.1.1],SizeBytes:8364694,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:52:42.685: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:42.730: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-z4ns Mar 16 14:52:42.786: INFO: up-down-2-z66f6 started at 2023-03-16 14:49:47 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container up-down-2 ready: true, restart count 0 Mar 16 14:52:42.786: INFO: success started at 2023-03-16 14:52:40 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container success ready: true, restart count 0 Mar 16 14:52:42.786: INFO: netserver-3 started at 2023-03-16 14:52:21 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.786: INFO: netserver-3 started at 2023-03-16 14:51:49 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.786: INFO: httpd started at 2023-03-16 14:52:36 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container httpd ready: false, restart count 0 Mar 16 14:52:42.786: INFO: pod-disruption-failure-ignore-3-n9wjf started at 2023-03-16 14:52:14 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container c ready: false, restart count 0 Mar 16 14:52:42.786: INFO: externalip-test-wtt65 started at 2023-03-16 14:52:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container externalip-test ready: true, restart count 0 Mar 16 14:52:42.786: INFO: pod-server-1 started at 2023-03-16 14:52:29 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.786: INFO: test-cleanup-deployment-68b75d69f8-dn66t started at 2023-03-16 14:52:39 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:42.786: INFO: metadata-proxy-v0.1-snr7k started at 2023-03-16 14:47:26 +0000 UTC (0+2 container statuses recorded) Mar 16 14:52:42.786: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:52:42.786: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:52:42.786: INFO: test-pod started at 2023-03-16 14:52:04 +0000 UTC (0+3 container statuses recorded) Mar 16 14:52:42.786: INFO: Container busybox-1 ready: true, restart count 0 Mar 16 14:52:42.786: INFO: Container busybox-2 ready: true, restart count 0 Mar 16 14:52:42.786: INFO: Container busybox-3 ready: true, restart count 0 Mar 16 14:52:42.786: INFO: slow-terminating-unready-pod-z7m62 started at 2023-03-16 14:52:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container slow-terminating-unready-pod ready: false, restart count 0 Mar 16 14:52:42.786: INFO: netserver-3 started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container webserver ready: false, restart count 0 Mar 16 14:52:42.786: INFO: kube-proxy-bootstrap-e2e-minion-group-z4ns started at 2023-03-16 14:47:25 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container kube-proxy ready: true, restart count 0 Mar 16 14:52:42.786: INFO: netserver-3 started at 2023-03-16 14:51:31 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.786: INFO: konnectivity-agent-r7xc5 started at 2023-03-16 14:47:26 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container konnectivity-agent ready: true, restart count 0 Mar 16 14:52:42.786: INFO: pod-disruption-failure-ignore-2-vxzkl started at 2023-03-16 14:52:10 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container c ready: false, restart count 0 Mar 16 14:52:42.786: INFO: pod-disruption-failure-ignore-0-g27z7 started at 2023-03-16 14:52:24 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container c ready: false, restart count 0 Mar 16 14:52:42.786: INFO: pod-disruption-failure-ignore-0-k6595 started at 2023-03-16 14:52:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container c ready: true, restart count 0 Mar 16 14:52:42.786: INFO: test-container-pod started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.786: INFO: pod-disruption-failure-ignore-1-cn28h started at 2023-03-16 14:52:02 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container c ready: false, restart count 0 Mar 16 14:52:42.786: INFO: httpd started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container httpd ready: true, restart count 0 Mar 16 14:52:42.786: INFO: execpod5q9fk started at 2023-03-16 14:52:28 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container agnhost-container ready: true, restart count 0 Mar 16 14:52:42.786: INFO: pod-disruption-failure-ignore-1-j2zz7 started at 2023-03-16 14:52:06 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container c ready: false, restart count 0 Mar 16 14:52:42.786: INFO: netserver-3 started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container webserver ready: true, restart count 0 Mar 16 14:52:42.786: INFO: test-rolling-update-with-lb-69fcddc54f-p8dgp started at 2023-03-16 14:52:07 +0000 UTC (0+1 container statuses recorded) Mar 16 14:52:42.786: INFO: Container agnhost ready: true, restart count 0 Mar 16 14:52:43.091: INFO: Latency metrics for node bootstrap-e2e-minion-group-z4ns END STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:52:43.091 (2.412s) < Exit [DeferCleanup (Each)] [sig-node] KubeletManagedEtcHosts - dump namespaces | framework.go:209 @ 03/16/23 14:52:43.091 (2.412s) > Enter [DeferCleanup (Each)] [sig-node] KubeletManagedEtcHosts - tear down framework | framework.go:206 @ 03/16/23 14:52:43.091 STEP: Destroying namespace "e2e-kubelet-etc-hosts-7419" for this suite. - test/e2e/framework/framework.go:351 @ 03/16/23 14:52:43.091 < Exit [DeferCleanup (Each)] [sig-node] KubeletManagedEtcHosts - tear down framework | framework.go:206 @ 03/16/23 14:52:43.161 (70ms) > Enter [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:52:43.161 < Exit [ReportAfterEach] TOP-LEVEL - test/e2e/e2e_test.go:144 @ 03/16/23 14:52:43.161 (0s)
Find test-pod, mentions in log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-node\]\sMount\spropagation\sshould\spropagate\smounts\swithin\sdefined\sscopes$'
[FAILED] failed to execute command in pod master, container cntr: error dialing backend: dial timeout, backstop: error dialing backend: dial timeout, backstop In [It] at: test/e2e/framework/pod/exec_util.go:107 @ 03/16/23 14:57:27.137
> Enter [BeforeEach] [sig-node] Mount propagation - set up framework | framework.go:191 @ 03/16/23 14:56:47.374 STEP: Creating a kubernetes client - test/e2e/framework/framework.go:211 @ 03/16/23 14:56:47.374 Mar 16 14:56:47.374: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename mount-propagation - test/e2e/framework/framework.go:250 @ 03/16/23 14:56:47.375 STEP: Waiting for a default service account to be provisioned in namespace - test/e2e/framework/framework.go:259 @ 03/16/23 14:56:47.508 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace - test/e2e/framework/framework.go:262 @ 03/16/23 14:56:47.593 < Exit [BeforeEach] [sig-node] Mount propagation - set up framework | framework.go:191 @ 03/16/23 14:56:47.702 (328ms) > Enter [BeforeEach] [sig-node] Mount propagation - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:56:47.702 < Exit [BeforeEach] [sig-node] Mount propagation - test/e2e/framework/metrics/init/init.go:33 @ 03/16/23 14:56:47.702 (0s) > Enter [It] should propagate mounts within defined scopes - test/e2e/node/mount_propagation.go:87 @ 03/16/23 14:56:47.702 Mar 16 14:56:56.660: INFO: ExecWithOptions {Command:[/bin/sh -c test -d /mnt/test/master] Namespace:mount-propagation-7916 PodName:master ContainerName:cntr Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 16 14:56:56.660: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:56:56.661: INFO: ExecWithOptions: Clientset creation Mar 16 14:56:56.661: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/mount-propagation-7916/pods/master/exec?command=%2Fbin%2Fsh&command=-c&command=test+-d+%2Fmnt%2Ftest%2Fmaster&container=cntr&container=cntr&stderr=true&stdout=true) Mar 16 14:56:56.962: INFO: Exec stderr: "" Mar 16 14:56:57.004: INFO: ExecWithOptions {Command:[/bin/sh -c test -d /mnt/test/slave] Namespace:mount-propagation-7916 PodName:master ContainerName:cntr Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false} Mar 16 14:56:57.004: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:56:57.005: INFO: ExecWithOptions: Clientset creation Mar 16 14:56:57.005: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/mount-propagation-7916/pods/master/exec?command=%2Fbin%2Fsh&command=-c&command=test+-d+%2Fmnt%2Ftest%2Fslave&container=cntr&container=cntr&stderr=true&stdout=true) Mar 16 14:57:27.136: INFO: Exec stderr: "" Mar 16 14:57:27.137: INFO: Unexpected error: failed to execute command in pod master, container cntr: error dialing backend: dial timeout, backstop: <*errors.StatusError | 0xc002506500>: error dialing backend: dial timeout, backstop { ErrStatus: code: 500 message: 'error dialing backend: dial timeout, backstop' metadata: {} status: Failure, } [FAILED] failed to execute command in pod master, container cntr: error dialing backend: dial timeout, backstop: error dialing backend: dial timeout, backstop In [It] at: test/e2e/framework/pod/exec_util.go:107 @ 03/16/23 14:57:27.137 < Exit [It] should propagate mounts within defined scopes - test/e2e/node/mount_propagation.go:87 @ 03/16/23 14:57:27.137 (39.435s) > Enter [AfterEach] [sig-node] Mount propagation - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:57:27.137 Mar 16 14:57:27.137: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready < Exit [AfterEach] [sig-node] Mount propagation - test/e2e/framework/node/init/init.go:33 @ 03/16/23 14:57:27.203 (66ms) > Enter [DeferCleanup (Each)] [sig-node] Mount propagation - test/e2e/node/mount_propagation.go:111 @ 03/16/23 14:57:27.203 Mar 16 14:57:29.454: INFO: ExecWithOptions {Command:[nsenter --mount=/rootfs/proc/1/ns/mnt -- sh -c rm -rf "/var/lib/kubelet/mount-propagation-7916"] Namespace:mount-propagation-7916 PodName:hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n ContainerName:agnhost-container Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true Quiet:false} Mar 16 14:57:29.454: INFO: >>> kubeConfig: /workspace/.kube/config Mar 16 14:57:29.455: INFO: ExecWithOptions: Clientset creation Mar 16 14:57:29.455: INFO: ExecWithOptions: execute(POST https://34.105.125.69/api/v1/namespaces/mount-propagation-7916/pods/hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n/exec?command=nsenter&command=--mount%3D%2Frootfs%2Fproc%2F1%2Fns%2Fmnt&command=--&command=sh&command=-c&command=rm+-rf+%22%2Fvar%2Flib%2Fkubelet%2Fmount-propagation-7916%22&container=agnhost-container&container=agnhost-container&stderr=true&stdout=true) < Exit [DeferCleanup (Each)] [sig-node] Mount propagation - test/e2e/node/mount_propagation.go:111 @ 03/16/23 14:57:30.102 (2.899s) > Enter [DeferCleanup (Each)] [sig-node] Mount propagation - test/e2e/node/mount_propagation.go:94 @ 03/16/23 14:57:30.102 STEP: Deleting pod hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n in namespace mount-propagation-7916 - test/e2e/framework/pod/delete.go:41 @ 03/16/23 14:57:30.102 < Exit [DeferCleanup (Each)] [sig-node] Mount propagation - test/e2e/node/mount_propagation.go:94 @ 03/16/23 14:57:30.198 (96ms) > Enter [DeferCleanup (Each)] [sig-node] Mount propagation - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:57:30.198 < Exit [DeferCleanup (Each)] [sig-node] Mount propagation - test/e2e/framework/metrics/init/init.go:35 @ 03/16/23 14:57:30.198 (0s) > Enter [DeferCleanup (Each)] [sig-node] Mount propagation - dump namespaces | framework.go:209 @ 03/16/23 14:57:30.198 STEP: dump namespace information after failure - test/e2e/framework/framework.go:288 @ 03/16/23 14:57:30.198 STEP: Collecting events from namespace "mount-propagation-7916". - test/e2e/framework/debug/dump.go:42 @ 03/16/23 14:57:30.198 STEP: Found 17 events. - test/e2e/framework/debug/dump.go:46 @ 03/16/23 14:57:30.269 Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:48 +0000 UTC - event for master: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:48 +0000 UTC - event for master: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:48 +0000 UTC - event for master: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:50 +0000 UTC - event for slave: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:50 +0000 UTC - event for slave: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:50 +0000 UTC - event for slave: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:52 +0000 UTC - event for private: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:52 +0000 UTC - event for private: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:52 +0000 UTC - event for private: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:54 +0000 UTC - event for default: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/busybox:1.29-4" already present on machine Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:54 +0000 UTC - event for default: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:56:55 +0000 UTC - event for default: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container cntr Mar 16 14:57:30.269: INFO: At 2023-03-16 14:57:27 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n: {kubelet bootstrap-e2e-minion-group-hzx6} Started: Started container agnhost-container Mar 16 14:57:30.269: INFO: At 2023-03-16 14:57:27 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n: {kubelet bootstrap-e2e-minion-group-hzx6} Created: Created container agnhost-container Mar 16 14:57:30.269: INFO: At 2023-03-16 14:57:27 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n: {kubelet bootstrap-e2e-minion-group-hzx6} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Mar 16 14:57:30.269: INFO: At 2023-03-16 14:57:27 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n: {default-scheduler } Scheduled: Successfully assigned mount-propagation-7916/hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n to bootstrap-e2e-minion-group-hzx6 Mar 16 14:57:30.269: INFO: At 2023-03-16 14:57:30 +0000 UTC - event for hostexec-bootstrap-e2e-minion-group-hzx6-s9x8n: {kubelet bootstrap-e2e-minion-group-hzx6} Killing: Stopping container agnhost-container Mar 16 14:57:30.332: INFO: POD NODE PHASE GRACE CONDITIONS Mar 16 14:57:30.332: INFO: default bootstrap-e2e-minion-group-hzx6 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:54 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:55 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:55 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:54 +0000 UTC }] Mar 16 14:57:30.332: INFO: master bootstrap-e2e-minion-group-hzx6 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:47 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:48 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:48 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:47 +0000 UTC }] Mar 16 14:57:30.332: INFO: private bootstrap-e2e-minion-group-hzx6 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:52 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:53 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:53 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:52 +0000 UTC }] Mar 16 14:57:30.332: INFO: slave bootstrap-e2e-minion-group-hzx6 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:50 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:51 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:51 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-03-16 14:56:50 +0000 UTC }] Mar 16 14:57:30.332: INFO: Mar 16 14:57:30.748: INFO: Logging node info for node bootstrap-e2e-master Mar 16 14:57:30.801: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master 38f900a3-5ef6-45a3-89cc-593197bf63fc 12046 0 2023-03-16 14:47:27 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2023-03-16 14:47:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2023-03-16 14:47:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2023-03-16 14:52:54 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jenkins-gce-gci-ip-aliases/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858370560 0} {<nil>} 3767940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596226560 0} {<nil>} 3511940Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2023-03-16 14:47:27 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:RouteCreated,Message:NodeController create implicit route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:27 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2023-03-16 14:52:54 +0000 UTC,LastTransitionTime:2023-03-16 14:47:29 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.40.0.2,},NodeAddress{Type:ExternalIP,Address:34.105.125.69,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jenkins-gce-gci-ip-aliases.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:7082699607a87ded236d2a4133f17058,SystemUUID:70826996-07a8-7ded-236d-2a4133f17058,BootID:17f7d8ba-9528-4ad5-ac76-f7ee18978020,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.6.6,KubeletVersion:v1.27.0-alpha.3.821+742316ee2136b2,KubeProxyVersion:v1.27.0-alpha.3.821+742316ee2136b2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:121701731,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:113649149,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:51eae8381dcb1078289fa7b4f3df2630cdc18d09fb56f8e56b41c40e191d6c83 registry.k8s.io/etcd:3.5.7-0],SizeBytes:101639218,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.3.821_742316ee2136b2],SizeBytes:59577838,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:b1389e7014425a1752aac55f5043ef4c52edaef0e223bf4d48ed1324e298087c registry.k8s.io/kas-network-proxy/proxy-server:v0.1.1],SizeBytes:21875112,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Mar 16 14:57:30.802: INFO: Logging kubelet events for node bootstrap-e2e-master Mar 16 14:57:30.858: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Mar 16 14:57:30.970: INFO: kube-apiserver-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container kube-apiserver ready: true, restart count 0 Mar 16 14:57:30.970: INFO: kube-addon-manager-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container kube-addon-manager ready: true, restart count 0 Mar 16 14:57:30.970: INFO: etcd-server-events-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:57:30.970: INFO: etcd-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container etcd-container ready: true, restart count 0 Mar 16 14:57:30.970: INFO: konnectivity-server-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container konnectivity-server-container ready: true, restart count 0 Mar 16 14:57:30.970: INFO: l7-lb-controller-bootstrap-e2e-master started at 2023-03-16 14:47:01 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container l7-lb-controller ready: true, restart count 2 Mar 16 14:57:30.970: INFO: metadata-proxy-v0.1-f5q72 started at 2023-03-16 14:47:27 +0000 UTC (0+2 container statuses recorded) Mar 16 14:57:30.970: INFO: Container metadata-proxy ready: true, restart count 0 Mar 16 14:57:30.970: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Mar 16 14:57:30.970: INFO: kube-controller-manager-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container kube-controller-manager ready: true, restart count 1 Mar 16 14:57:30.970: INFO: kube-scheduler-bootstrap-e2e-master started at 2023-03-16 14:46:45 +0000 UTC (0+1 container statuses recorded) Mar 16 14:57:30.970: INFO: Container kube-scheduler ready: true, restart count 0 Mar 16 14:57:31.224: INFO: Latency metrics for node bootstrap-e2e-master Mar 16 14:57:31.224: INFO: Logging node info for node bootstrap-e2e-minion-group-2p9b Mar 16 14:57:31.284: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2p9b 36bf5a06-b512-4a74-9b11-3047be99f97b 22634 0 2023-03-16 14:47:24 +